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PREFACE 


The purpose of this volume is to make available the results of work that 
has been done during the past decade in the Psychometric Laboratory at 
the University of Chicago. The material is presented in the form of a text 
for students. Essentially all the material of the first edition, The Vectors of 
Mind, has been retained here, and much has been added both in multiple- 
factor theory and in computational method. Detailed computational office 
routine has not been featured because it will be covered in a workbook on 
multiple-faetor analysis. 

The Vectors of Mind has been out of print twice, and the current reprint- 
ing is now exhausted. When the writingof the present volume was started, 
it was my intention to revise The Vectors of Mind, re-writing certain sec- 
tions and making additions of new material. This plan was altered by 
adopting a more complete expository style for the present text. The entire 
book has been re-written, and hence it has been given a new title. The Vec- 
lors of Mind will be referred to in this text as the “first edition." 

It has not been my intention to produce a bibliographically complete 
treatise on multiple-faetor analysis. That would be entirely beyond the 
scope of the present text. Only those references have been given that were 
direetly in line with the subject of discussion, but I fear that even with this 
limitation I have not done justice to all the authors whose work should 
have been summarized in this text. Most of these references are readily 

available in the journal Psychometrika and in the bibliographies of its many 
papers on factorial theory. The subject is still advancing rapidly, so that a 
theoretically complete treatise would become obsolete in a few years. When 
the time comes for such a volume, it should be written by a mathematician 
or by an author who is competent to appraise the contributions of mathe- 
maticians and physicists in this field. 

The brevity of The Vectors of Mind was probably responsible for much 
of the controversy about multiple-factor analysis. As far as I am aware, all 
the theorems in the first edition are as valid today as they were ten years 
ago. The theorems were stated, and, in general, they were proved; but I 
left them without the support of exposition as to their implications and in- 
terpretation in exploratory scientific work. Several concepts have been the 
subject of controversy. It has been a mystery to me why the f undamentally 
simple notion which I called “simple structure” has been twisted around 
by some critics into the most inconceivable nonsense. There has been 
misunderstanding about the communality concept by which multiple- 
factor analysis is limited to the common factors—Jjust as it is in Spearman's 
single-factor case. Some students of this subject change their set in going to 
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the multiple-factor case by insisting that they must then include the total 
test space. There has also been confusion about the oblique reference frame 
which represents parameters or factors that are correlated in the general 
population as well as in the experimental population. 

When multiple-factor analysis was introduced in 1931,* it constituted a 
shift of emphasis. Papers being published at that time were mainly con- 
cerned with the existence of Spearman’s central intellective factor g, the 
tetrad difference criterion for the general factor g, probable error formulae 
for the tetrad difference, and group factors which were regarded as the dis- 
turbers of the general factor and which were nearly always regarded as 
residuals that were secondary to the general factor, There is nothing new 
in the recognition that more than one factor must be postulated to account 
for the observed test correlations. Instead of asking whether the central 
intellective factor g can be demonstrated in any given set of correlations, 
with or without disturbing group factors, we ask how many factors are in- 
dicated by the correlations without restriction as to whether they are gen- 
eral or group factors. Instead of extracting, first, the postulated general 
factor and then investigating the residuals to determine whether any dis- 
turbing group factors must also be admitted, we start with an observation 
equation in n terms which represent, as many factors as may be required 
by the correlations. It becomes, then, a question of fact as to whether one 
or more of them are general faetors and whether one or more of them are 
positive or bipolar. 

Our matrix formulation of the problem led to the fundamental faetor 
theorem FF’ = R, where F is a factor matrix and R is the correlation ma- 
trix. The matrix formulation of factor analysis seems to have been generally 


accepted, and it has largely replaced the previous methods in factor 
analysis. 


In 1931 I decided to investigate the relation between multiple-factor 
analysis and Spearman’s tetrad differences. When I wrote the tetrad equa- 
tion to begin this inquiry, I discovered that the tetrad was merely the ex- 
pansion of a second-order minor, and the relation was then obvious. One 
might speculate as to whether multiple-factor analysis would have de- 
veloped earlier if this interpretation had been stated earlier. If the second- 
order minors must vanish in order to established a single common factor, 
then must the third-order minors vanish in order to establish two common 
factors, and so on? To have put the matter in this way would have led to 
the matrix formulation of the problem much earlier, as well as to the im- 
mediate development of multiple-factor analysis. Instead of dealing with 
the proportional columns and rows of a hierarchy and the vanishing tet- 
rads, we now deal with the same relations in terms of the properties of unit 
rank, namely, proportional columns and rows and vanishing second-order 
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minors. This formulation generalizes to the properties of higher rank. As 
far as Lam aware, this formulation of the factor problem has been generally 
accepted. 

The test correlations define a configuration of test vectors without a ref- 
erence frame. Since the factors are represented by the axes of this frame, it 
is necessary to locate a reference frame somehow in the test configuration. 
Here we have to recognize a distinction between an arbitrary orthogonal 
reference frame that is used for computing purposes and the reference 
frame that should be used for scientific interpretation of the factors. The 
arbitrary orthogonal frame is defined by the method of factoring that the 
computer happens to use. In order to locate a reference frame for interpre- 
tation, I used, at first, a statistical eriterion which led to the principal axes 
of the configuration; but I soon discarded this solution in favor of a prin- 
ciple that I called “simple structure." This principle of simple structure 
has been the cause of much controversy, and it is not yet accepted by all 
students of factorial theory. The fact that it very frequently gives a set of 
factors or parameters that can be interpreted as meaningful in scientific 
context would seem to be an argument in its favor, but even that result 
has been the object of controversy. I have found that the best way to con- 
vince а skeptic is to let him work with the rotational methods on a prob- 
lem in which he is interested. As the reference frame begins to emerge in 
the rotational work with successively clearer interpretation, he convinces 
himself of the power of the method. 

In locating a meaningful reference frame for a set of correlated variables, 
one may voluntarily impose the restriction that the reference frame shall 
be orthogonal, or one may allow it to become oblique according to the test 
configuration. If we impose the restriction that the reference frame shall 
be orthogonal, then we are imposing the condition that the factors or 
parameters shall be uncorrelated in the experimental population or in the 
general population. It is my own conviction that this restriction should 
not be imposed if we are looking for meaningful parameters. It seems just 
as unnecessary to require that mental traits shall be uncorrelated in the 
general population as to require that height and weight be uncorrelated in 
the general population. This admission of oblique axes has also been de- 
bated and is not yet generally accepted by students of factor theory. 

The correlations among primary factors must be expected to vary from 
one experimental population to another, depending on selective conditions, 
This does not mean that the physical nature of each factor or parameter 
has altered. Just as the correlation between height and weight varies from 
one group to another by selective conditions, so the correlations between 
mental traits may also vary from one group to another without change in 


their meaning. i 
Second-order factors which are determined from the correlated primaries 


have only recently been introduced, so that it is not yet known how fruitful 
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they will be. It seems likely that a second-order gencral factor, determined 
from correlated primaries, may turn out to be Spearman’s general intellec- 
tive factor g. Spearman’s early admonition that the tests in each tetrad 
must not be very similar may be the equivalent of our present observation 
that aset of four tests can be selected so as to involve only one primary fac- 
tor for each test. Such a set of four tests will have only the second-order 
general factor in common; hence the correlations will be of unit rank, and 
the second-order minors must then vanish. When two similar tests are in- 
cluded in a tetrad, they are likely to involve the same primary as well as 
the second-order general factor, and the unit rank is then disturbed. 

In the scientific interpretation of factorial results it is necessary to dif- 
ferentiate between what we have called “configurational invariance” and 
the numerical invariance of correlations and saturation coefficients. In 
ant, since it transcends 
most selective conditions for the groups of experimental subjects. Those 
authors who insist on avoiding geometrical interpretations in factor anal- 
ysis will almost certainly have some difficulty in dealing with configura- 
tional invariance and its important implications. The same results can 
usually be stated in analytical form, but it is likely that the geometrical 
ideas will precede the analytical forms. 

Since the publication of The Vectors of Mind, there have been several 
texts on factorial theory. The text by Holzinger and Harman presents fac- 
tor analysis as a statistical method.* This is a worth-while objective, but 
the mathematical statisticians have not yet accepted factor analysis as a 
part. of statistical logic. Holzinger describes several factoring methods, in- 
cluding his own bi-factor method. The simple-structure concept is disposed 
of in a footnote, but the rotational problem is presented in variant forms 
of factorial resolution. It is unfortunate that the term “structure” has been 
defined in different ways. Holzinger accepts the matrix formulation of the 
multiple-factor problem and the interpretation of Spearman’s single-factor 
hypothesis as a special case of unit rank. The oblique axes are described 
as alternative solutions. 

Truman L. Kelley’s monograph, Essential Traits of Mental Life, de- 
seribes an ingenious method of arriving at the principal-axes solution di- 
rectly from the correlation matrix. Kelley docs not accept the communali- 
ties, or the simple-structure concept, or rotation from the principal axes 
for the purpose of isolating simpler and more meaningful parameters. He 
attempts to interpret directly the principal axes of each test battery. Kelley 
insists that factor analysis must be made with unit diagonals to represent 
the total variance rather than the common-factor variance. His earlier 


general, configurational invariance is more signific 


* Karl J. Holzinger and Harry H. Harman, Factor Analysis ( 


n" Chicago: University of 
Chicago Press, 1941). 


1 Cambridge: Harvard University Press, 1935. 


PREFACE ix 
. 
monograph, Crossroads in the Mind of Man,* should also be acknowledged 
as a significant contribution. In that publication he attempted to break 
away from the dominant single general factor to the multiple-factor prob- 
lem, but he did not then introduce the generalized factor problem. 

Burt f accepts the communality concept, the matrix formulation of fac- 
tor analysis, and the interpretation of Spearman’s single-factor hypothesis 
as a special ease of unit rank, but he does not accept the necessity for ro- 
tating the reference frame for purposes of interpretation. Neither does he 
accept the use of oblique reference axes. The configurational interpreta- 
tions are evidently distasteful to Burt, for he does not have a single dia- 
gram in his text. Perhaps this is indicative of individual differences in 
imagery types which lead to differences in methods and interpretation 


among scientists. 

The most provocative of the recent texts in factor theory is that of God- 
{теу H. Thomson. Even when he differs from our interpretations of the 
factor problem I find his discussion stimulating for our own work. Our 
chapter on selective conditions and sections of the chapter on unit rank 
resulted directly from the stimulus of his provocative papers. He does not 
seem to be convinced about the necessity for rotating the reference frame, 
although he is probably open-minded about the question. He is not opti- 
mistie about the interpretation of primary factors as fundamental parame- 


ters in the description of mentality. 

While there has been considerable controversy about multiple-factor 
analysis, it is probably true that the large majority of factorial studies in 
recent. years have accepted the matrix formulation of the problem, the 
interpretation of Spearman's single-factor theorems as a special case of 
unit rank, the communalities, the simple-structure concept, the necessity 
for rotating the reference frame for scientific interpretation, the desira- 
bility of interpreting the primary factors as meaningful parameters, the 
use of oblique reference axes, and the principles of configurational in- 
variance. 

No attempt has been made here to review the numerous applications of 
factor analysis to psychological and other problems. Dael Wolfle’s mono- 
graph, Factor Analysis to 1940,8 was a good summary of theory and of 
dions, and such reviews should be made periodically either by new 
monographs like Wolfle's or by periodic summaries in the Psychological 
Bulletin. Vor several reasons in the present text I have used very few psy- 
est examples. When psychologists start to work with psycho- 


applic: 
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logical test correlations to find primary mental abilities, they sometimes 
jump the track and fall into a discussion about faculty psychology, The 
logic of the subject can be presented perhaps more clearly by using physical 
objects. That is the reason we have used populations of rectangular boxes, 
populations of cylinders, and a population of trapezoids. Here the student's 
attention is directed to the interpretation of factors as parameters in the 
usual scientific sense. In this way we attempt to make the relations of the 
parameters in factorial representation as vivid as possible. 

One of the limitations of factor analysis in its present stage of develop- 
ment is the absence of sampling-error formulae. It is perhaps natural that 
factorial logic should be developed before the sampling-error formulae. It 
is only recently that the influence of selection of tests and of experimental 
subjects on factorial results have been investigated. These effects must be 
distinguished as to the conditions that produce stable numerical values 
and those in which configurational invariance is of primary interest. The 
relations of second-order factors to factorial results have also only recently 
been investigated. At present we are limited largely to the empirically ob- 
served stability of factorial findings. The availability of sampling-error 
formulae for the principal-axes solution for a given correlation matrix is 
good as far as it goes, but this is a trifle compared with the sampling- 
error problems in factor analysis. 

Factor analysis has not been generally accepted by mathematical statis- 
ticians. The present conceptual formulations of statistical theory do not 
seem to lend themselves readily to the logical problems of factor analysis. 
One might resolve this difficulty in one of two ways. One might insist that 
multiple-factor analysis be re-written within the restrictions of current 
statistical theory, or one might ask the statisticians to adapt and expand 
their formulations to cover the logical problems of factor analysis. Perhaps 
both ways will be tried. 

When the mathematical statisticians eventually decide to assist the stu- 
dents of multiple-factor analysis, it will be helpful if certain fundamental 
considerations are recognized at the outset. Some of these can be listed 
here. First, let us recognize that when a scientist is trying to invent a theory 
to cover the phenomena of a particular domain, a simple description of its 
underlying order, he is certainly not concerned with sampling distributions. 
Furthermore, he must restrict himself to some aspect of the phenomena in 
the domain. He cannot hope to cover the total variation, for no scientific 
formulation ever covers the total variance in any domain. When the statis- 
tician turns his attention to the multiple-factor problem, he must deal 
with the communality concept and its effect on the rank of the correlation 
matrix. If he denies this problem, he not only admits his inadequacy but 
also his failure to understand the problem. If statistical theory is not now 
adequate to cope with the communalities, then the statistician has the 
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challenge to adapt his formulations to the limitations that are inherent in a 
very large class of scientific problems. He need not deal with the commu- 
nalities as such because he may discover some equivalent resolution, but he 
will not solve the problem by laughing it out of court. 

Another consideration with which the statistician is already familiar is 
the distortion introduced by the assumption of linearity in the exploratory 
study of a new domain. Treatises in the more advanced sciences abound 
with non-linear relations. Linear relations are the exception. There is no 
good reason to believe that relations in the younger sciences are funda- 
mentally simpler. When scientific inquiry enters upon a new domain with 
hazy ideas about the underlying order or, perhaps, with no ideas at all, the 
scientist is as naive as we would be if we should make an experimental in- 
quiry about the relation between the sizes and the weights of boxes with- 
out knowing their contents. He would measure and weigh the first one hun- 
dred boxes and determine the correlations and the regression equations, 
and then he might worry about the standard errors of his coefficients. 
Something would be gained by doing that, because it would affect the de- 
gree of assurance with which he could say something about the relations 
that he had found. If the scientist takes his numerical coefficients very 
seriously at the exploratory stage, he may be lacking in a desirable sense of 
humor about. the erudeness of all his tools in spite of their polished appear- 
ance. Too much concern with numerical minutiae at that stage may lead 
him astray from the conceptual formulations that constitute his real goal. 
If factor analysis is one of his tools for the exploration of a new domain, his 
principal concern will be with the factor pattern, the presence or absence 
of a significant coefficient in each cell of the oblique factor matrix, rather 
than the numerical values of any of them. This consideration may simplify 
the statistician’s problem in factor analy: 8. Tun | i 

So far, one of the principal handicaps of the statistician ind aling with 
multiple-factor analysis has been his failure to appreciate the significance 
of the rotational problem. This is the problem of choosing the most fruitful 
of parameters to describe the variation in a domain. These parameters 
represent scientific concepts. They are not merely numerical coefficients, 
If these scientifically meaningful parameters happen to be correlated in 
the experimental sample, such as height and weight, that is no reason for 
tefeoting them. The prior consideration is to find fruitful and meaningful 
parameters The question of whether the cross products vanish 15 a trivial 
consideration of the computer's convenience. The statistician must under- 
stand this problem of oblique reference axes and associated scientific con- 
cepts. The statistician must not Impose the arbitrary restriction that our 
scientific concepts in the description of people must be uncorrelated either 
in the experimental sample or in the general population. 
attempting to introduce sampling distributions in factor analysis, it 
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will also be useful to recognize that when a simple structure is discovered in 
a new domain, it is rarely equally stable or clear in all the dimensions. Some 
of the hyperplanes may be overdetermined while others are even indeter- 
minate. These hyperplanes are usually unknown at the start of the experi- 
ment, and no statistical jugglery can evade their different degrees of over- 
determination unless the jugglery merely obscures them. 

One of the most useful conceptual formulations in statistical theory is 
that of the sample and the universe from which the sample is drawn. A 
large part of statistical theory is built on this construct. This often leads to 
a concern as to whether any given sample is representative of a postulated 
universe, One might suppose in factor analysis that the experimental sam- 
ple should be representative of the general population, but such is not. the 
case. On the contrary, if a certain group of traits is to be investigated as to 
their underlying structure, we proceed as in many other types of scientific 
investigation by selecting a highly biased sample, including even freaks in 
the domain concerned. Another sample of freaks may be factorially studied 
without any question as to whether it is also an unbiased sample from the 
same universe. Such studies are much more likely to be revealing of the 
underlying nature of the domain than are carefully randomized samples 
from the general population. And, further, there is no necessity that the fre- 
quency distribution of anything shall be Gaussian. Let us also not forget a 
simple principle that every scientist takes for granted, namely, that he 
would rather measure something significant without any sampling distribu- 
tions than measure something trivial or irrelevant because its sampling 
distribution is known. 

In arecent paper Hotelling deplores the fact that students of factor anal- 
ysis have not made use of the recent advances in mathematical statistics, 
and he dismisses the fundamental concepts of multiple-factor analysis be- 
rause they have not been formulated in terms of current statistical theory. 
A fair appraisal of this situation would probably acknowledge that we 
should have succeeded better in getting the co-operation of the statisticians 
if recent advances in statistical theory had been recognized explicitly; but 
it would probably also acknowledge that, while statistical the 
important advances in the last two decades, it has not advanced far enough 
to be immediately useful for factor analysis as a scientific method without 
still further advancement. Instead of dismi ng the basic concepts of 
multiple-factor analysis, we must reach common ground in recognizing 
that these concepts represent difficult problems in the logie of science that 
need to be studied rather than ridiculed. Let us hope that competent stu- 
dents will begin soon to invest igate these challenging problems, so that the 
factorial methods ean be made generally available in the sciences without 
the handicap of distracting controversy. 


sory has made 
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A NOTE ON FUTURE WORK 

In presenting this text as à summary of our work on multiple-factor 
analysis, is may be appropriate to mention some possibilities for the future 
development of this subject. One of the principal assumptions underlying 
factorial theory is that the scores are monotonic increasing or decreasing 
functions of the scores on the primary factors or parameters. The funda- 
mental observation equation of factor analysis makes the further assump- 
tion that these functions can be expressed in linear form as a first approxi- 
It would be possible to start with a second-degree observation 
to develop factorial methods on that basis. Instead of devel- 
oping factorial theory more completely with observation equations of 
higher degree, it would probably be more profitable to develop non-metric 
methods of factor analysis. An idea for such a development would be to 
determine the number of independent parameters of a score matrix by 
successive differences in rank order on the assumption that they 
tions of a limited number of independent parameters, N 
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score would then be regarded as merely an index of rank order, and that is 
essentially what we are now doing. The raw scores are transmuted into a 
normal distribution of unit standard deviation, and these transmuted 
scores are used for the correlations. Instead of dealing with the transmuted 
scores in this manner, one might deal with the rank orders directly or with 
some equivalent indices of rank order. Analysis of successive differences in 
rank order, or their equivalents, should lead to the factor pattern, which 
would indicate the number of parameters involved in the variance of each 
test. That is the main object of factor analysis. The actual numerical values 
are of secondary importance in teasing out what we have called the under- 
lying order of a new domain. The simple-structure concept is applicable 
as well to a non-metrie form of factor pattern as to the more restricted 
linear metric form that we have been using. The scientific interpretation 
of the factors could be made from a factor pattern without putting it in 
metric form. If the mathematical theory has already been developed, it 
would be helpful if we knew what to call it in asking mathematicians for 
assistance. In putting the results to practical use, the problem would return 
to a metric form, with standardization and norms which call for statistical 
methods of the conventional kind. The main problem for factor analysis 
is the more fundamental one of charting the alternative factor patterns in 
new fields. To develop this subject with more powertul non-metric methods 
is future work that may leave our present efforts obsolete. 


L. L. THURSTONY 
CHICAGO, ILLINOIS 
March 1945 
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MATHEMATICAL INTRODUCTION 


The matrix theory which is used in the development of factor analy 
not generally available to students whose training in mathematics is limited 


is 


to undergraduate courses in analytical geometry and in the ealeulus. This 
mathematical introduction reviews the elementary theory of matrices as 
well as the closely related theory of determinants. Summation that involves 
double subseript notation is included in this section, since it is used in factor 
theory and since it is unfamiliar to most students of statistics. In the geo- 
metrical interpretation of the factorial matrix, only non-homogeneous co- 
ordinates are used. For this reason, the introduction includes non-homo- 
geneous co-ordinates and omits homogeneous co-ordinates which are con- 
ventional. Orthogonal and oblique transformations have been illustrated 
geometrically. No provocation has been found so far in factor theory to in- 
troduce imaginaries and complex numbers, but the future development of 
factor analysis may call for them. This mathematical introduction is limited 


to the real case, and all theorems have been written with this restriction in 


mind. 

If this introduction is not self-sufficient, perhaps it may serve as a useful 
guide to the student of factor theory who seeks mathematical assistance on 
specified topies. Ifa student has the intention of attaining some competence 
in factor theory and in related statistical work, there is no short cut for 
formal courses in the mathematies that is involved.* 


Matrices 

Matrices and determinants involve rectangular arrangements of num- 
bers. Any rectangular arrangement of numbers is called a matrix, irrespec- 
tive of what the numbers mean. If the matrix has m rows and n columns, 


the matrix is said to be of order m XR. In designating the order of a matrix, 


vill be found useful: 


* The following references v | eu 
traustein, Plane and Solid Analytic Geometry (New York: 


W. F. Osgood and W. C. G 


Maemillan Co., 1929). Р D , | 1 
L. E. Dickson, Modern Alge braic Theories (New York: B. Н. Sanborn, 1926), chap. iii. 


Maxime Bocher, Introduction to Higher Algebra (New York: Macmillan Co., 1931), 


chaps. i-vi, ine. 


H. W. Turnbull and А. C. Aitken, Canonical Matrices (London: Blackie & Sons, 


932), chap. i. du 
: = RE and C. H. Sisam, A nalytical Geometry of Space (New York: Henry Holt & 
Co., 1014). 

A. Adrian Alb 


1941). ик Е 
W. L. Ferrar, Algebra (Oxford University Press, 1941). А 
kenali Dresden, Solid Analytical Geometry and Determinants (New York: John Wiley 


& Sons, 1930). ї 


ert Introduction to Algebraic Theories (Chicago: University of Chicago 


Pre 
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it is customary to refer to rows first and columns st cond. Thus a matrix of 
order pq has p rows and у columns. Tables fa and /b show a matrix of 
3x4 and a matrix of order 3X3. A row is horizontal. A column is 
vertical. The general name for either a row or a column is an array. Each 
of the small squares into which a matrix is divided is called a cell, and the 
number in each cell is called a cell entry or clement. 


order 


Table Та Table 1b 
2 у 1 8 rp 3 
|o 9 [5.3 
0 E eT Bx т 


In order to designate a particular clement, it is customary to use a double 
subseript, the first one for the row and the second one for the column. If a 
matrix is denoted A, then its elements may be denoted а, where i shows 
the row and j shows the column at the intersection of which the element aij 
is found. Thus, in Table Га the clement ay2=3 and а;=9. 

In developing the theory of matrices it is desirable to exhibit the ele- 
ments as shown in Table 2. The elements in the first row are au, ai, 13, 

2, Mn, showing that the table represents a matrix of n columns. The elc- 
ments of the first column are ay, аш, “31, ... Gna, Showing that the table 
represents m rows. The general element in this matrix A is а, where 7 
takes the successive values 1, 2, 3 


, 2, 3,..., m, while 7 takes the successive 
values 1, 2,3,..., n. The first subscript refers to the row; the second sub- 
script refers to the column. 


Table 2 


On hs “з >.. Min 

ün аз (ag s.. (lan 

da du üs ... rm 
llii 

Ami uz Ams ... Omn 


The conventional representation of a matrix is shown in Table 3, where 
the rectangular arrangement of numbers is inclosed by double vertical lines 
on the left and on the right sides of the rectangle. It is also customary to 
denote specified matrices with letters. Thus, the matrix of Table 3 might be 
conveniently designated A or any other letter. A matrix A might also he 
designated by its general element aij. 


Table 3 


2.1 &]| 


14 8 3 
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If the successive rows of matrix A are written as successive columns of a 
new matrix, the new matrix is called the transpose of A. It is denoted A’. 
Table 4 shows a matrix A and its transpose P ds 


Table 4 
9 d Lu up E 
LOYS 3.0 9 
0 2 6 7 I quud 
0 9r 
A A 


Determinants 

One particular interpretation of a square matrix is called a determinant. 
This interpretation of a square matrix probably had its origin in the practi- 
eal work of solving simultaneous equations, and it is indieated by single 
vertical lines on the left and on the right sides of a square table. It is illus- 
trated in Table 5. Table 3 is called a matrix; while Table which implies a 
particular interpretation, is called a determinant, A determinant is always 
square. Hence its order is n, in which n is the number of rows or the number 
of columns. 


Table 5 


2. 1.5 
48 3 
207 


The diagonal from the upper left corner to the lower right corner of a 
determinant is called the principal diagonal. In Table 5 the principal diag- 
onal contains the elements 2, 8, 7. The other diagonal from the lower left 
corner to the upper right corner is called the secondary diagonal. 

In many problems it is convenient to assign a plus sign and a minus sign 
to alternate cells in à determinant. A convenient rule is to designate the 
upper left, cell as positive and all other cells as alternately negative and 
positive, as the cells ean be moved over by the castle in a chess game. This 
ated in Table 6 for a determinant of order 5. 


sign arrangement is illustr: 
Table 6 
+ - 4- - т Ж 
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Notice that the cells in the principal diagonal are all positive and that lines 
parallel to this diagonal are alternately negative and positive. The sign is 
positive for an even number of steps from the upper left cell, and it is nega- 
tive when the number of steps is odd. The sign of a cell determined in this 
manner may be ealled the position sign of the cell. If the general element of 
the determinant is denoted аз, then the element with its position sign may 
be conveniently denoted (= 1) аз. When the exponent (4-7) is odd, the 
sign of the cell is negative; and when (743) is even, the sign of the cell is 
positive. 

The product of any n elements of a square matrix, selected with only one 
clement from each of the » rows and only one element from each of the н 
columns, is called a term of the determinant of the matrix. Table 7 is a deter- 


Table 7 


ап ап а 


minant of order 3. From this determinant six terms may be written. ‘These 
are shown in Table 8, in which the elements of each term are arranged in 
the order of their columns. 
Table § 

1) an an 35 
) ац Az 025 
3) an ар az 
1) an аз 01:3 
5) ах (a 025 


6) an An аз 


Each of these six terms is the product of three elements so selected that 
each term contains only one element from each row and only one element 
from cach column. If a square matrix is of order n, the total number of 
terms in its determinant is n. The term that contains all the elements of 
the principal diagonal is called the leading term of the determinant. 

The sign of each of the n terms of a determinant can be ascertained in 
the following manner. Let the » elements of each term be arranged in as- 
cending order according to columns, as shown in Table 8. This can evident- 
ly be done without affeeting the numerieal value of the terms. Consider 
the fourth term as an example, and list the rows as follows: 
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Any interchange of two adjacent elements constitutes an inversion. It may 
be illustrated by interchanging 1 and 3. The resulting arrangement is 


2 1 3. 


If, now, the adjacent elements 1 and 2 are interchanged, the arrangement 


becomes 
m x 
i 2 5, 


in which the rows are in consecutive order. 

The sign of a term of а determinant is positive if it represents an even 
number of inversions from the consecutive order of rows and columns. The 
ative if the number of inversions in the term is odd. 


sign of the term is neg, 
rms of Table 8, we have the same terms 


Applying this rule to the six te 
with proper signs as shown in Table 9. 


Table 9 
1) Fan аза» 
2) -—amn аваз 
3) —ал аз аз; 
4) +an аз аз 
5) +431 dis Az 


6) —@s1 d2 Qs 


A complete definition of a determinant can now be given. 

Definition: If a square table is used as a symbol of the sum of n terms, 
cach term being the product of n elements with only one element from 
cach row and only one clement from each column, the sign of each term 
taken positive or negative according as the term contains an even or an 
odd number of inversions, then the square table is called a determinant. 

Hence the sum of the six terms of Table 9 is implied by the determinant of 
Table 7. The determinantal interpretation of a square matrix is denoted by 
single vertical lines on the left and on the right sides of the square table, as 
shown in Table 7. 

If a square matrix is denoted by a letter such as A, then the determinant 
of the matrix is denoted [A]. If A represents a number, then |A| means 
the absolute value, ignoring the sign of the number A. It should be noted 
that a matrix is merely а rectangular table of numbers, and hence a matrix 
a determinant is, by definition, a sum of terms, 


has no numerical value. But y ‹ 
atrix is denoted a;;, then its de- 


and hence it has à numerical value. If a m 


terminant is denoted lail- 
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Consider the second-order determinant 


and the 2=2 terms that it implies. These are 1X3 and 8X5, in which the 
factors of each term are arranged in consecutive order by columns. The rows 
of the term 1X3 are 1 and 2. Since these are in consecutive order, the sign 
of this term is positive. Its value is therefore 4-(1)(3) = +3. The rows of 
the term 8X5 are 2 and 1. One inversion changes the order 2 and 1 into the 
consecutive order 1 and 2. Hence the sign of this term is negative. The de- 
terminant therefore has the numerical value +3—40=—37. Any second- 
order determinant can be evaluated as follows: 


= ab — cd. 


Ап z-rowed minor of the matrix A is a determinant of order x which is 
formed by the intersections of any x rows and any 2 columns of the ma- 
trix A. If one or more columns of a determinant are eliminated and if the 
same number of rows are eliminated, the remaining cells constitute a minor. 
From the determinant of Table 7 nine second-order minors may be drawn. 
A few of them are illustrated here: 


ап a аз (23 (12 13 


а da2 азр 433 аз» 


(23 


If any two columns and any two rows are eliminated from the determinant 
of Table 7, there remains a 1-rowed minor which is a single element. In this 
sense each element can be regarded as a minor of the determinant. 

If corresponding rows and columns are eliminated, the remaining minor 
is symmetrically placed with regard to the principal diagonal, and it is 
called a principal minor. In the determinant of Table 7, three second-order 
principal minors may be drawn. These are 


ап а, а аз da2 da3 


азу da 


t$ 


азі d33 a32 033 


There are three 1-rowed principal minors in this determinant, namely, the 
three elements in the principal diagonal. 
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If the row i and the column j which intersect in an element a;; are elimi- 
nated from a determinant, the remaining (n—1)-rowed determinant is 
called the first minor of а: This definition is illustrated with the determi- 
nant of Table 5. The second row and the first column intersect in the ele- 
ment 4. If these two arrays are eliminated from the determinant, the re- 


maining 2-rowed determinant is 


p 5 
=7-0= +7. 
| 0 


Т 
This determinant, whose numerical value is +7, is the first minor of the 


element аз =4 in the determinant of Table 5. Let the first minor of the ele- 


ment a;; be denoted т; 
In some problems it is convenient to refer to the minor mi; with the posi- 


tion sign of the element ai; This quantity is called the cofactor of a;;. It is 


defined by the relation 


(cofactor of ajj) = ei; = (— 1) mij. 


In Table 5 the cofactor of the element 4 is 


EM 


(—1y" | = — [7-0] = – 7. 


B. y 
In the same table the cofactor of the element 3 is 


2 1 


(—1) = – [0-2 = +2. 


Hence the absolute values of the first minor of a;; and of its cofactor are 
identical. They differ only in the manner of determining the sign. If the 
position sign of the element a;; is positive, the first minor and the cofactor 


have the same sign. If the position sign of aj; is negative, they have op- 


posite signs. 
The numerical value of a determinant can be expressed conveniently for 


some problems in terms of the cofactors. For example, 


ап d: — dis 


азу а  Q23 = апеп + aaea + azez. 


азу @32 @зз 


8 MULTIPLE-FACTOR ANALYSIS 


The numerical value of a determinant is the weighted sum of the elements in 
any array, each element being weighted by its cofactor. In the example, the 
determinant is expressed in terms of the elements of the first column. 

As a numerical example, the value of the determinant of Table 5 can be 
expressed as follows: 


2 9| 
| | |8 3] @ 55, o5 
SOT gd | —4 +2 | 
| [К О ОУУ E 3.) 
E O +7] 


= 2(56—0) — 4(7—0) + 2(3—40) 
112 — 28 — 74 = 4- 10. 


The numerical value of a determinant can be expressed as the summa- 
tion: 


n n 
1 4 ^ 

(1) [а= X апе; = P апе, 
i-1 j=l 


where the weighted sum may be taken over any column or any row. The 
following is an example of a fourth-order determinant, evaluated by the 
method of (1): 


2410 
3242 242 410 410 410 
141= [1614 | =+2| 614 |-3| 614 |+1| 242 | -1 242 
1023 023 023 023 614 
242 ia 
esiti peii 
023 $ s 
= 2(3—8) — 6(12—4) + 0(16—2) 
= 10 — 48 +0 = — 58. 
410 
14 10 10 
614 |—-44 —6 +0 
23 23 14 
023 


4(3—8) — 6(3—0) + 0(4—0) 
= = 20-18 +0 = —38. 
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410 ; 
4 10 10 
242| 244 5 
2 23 42 
023 
= 4(12—4) = 2(3—0) + 0(2—0) 
-32—6--0- + 26. 
J410 
42 10 10 
242| =+4 -2 
14 14 42 
[ota 
= 4(16—2) — 2(1—0) + 6(@—0) 
= +56-8+12= + 60. 


Hence 


|А| = (+2)(—58) — (8)(—38) + (6-26) — (1)(+60) = — 36. 


For ever 


ry element aij in the square matrix A there is a corresponding 


minor m;; and a corresponding cofactor е;;. Let M be the square matrix 
ij ° 


with elements ту; let Æ 
the transpose of Û. 


Its elements may be denoted Sig = esi 


These defini 


o; 1 5 
4 8 3] =4 
ОТ 


= М 


56 


—22 


—22 


—16 


2 


12 


—37 


14 
12 


be the square matrix with elements е;;; and let F be 
Then the square matrix 7 is called the adjoint of A. 


tions are illustrated in the following numerical example: 


=F =adjoint of A. 
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A square matrix A is said to be symmetric when A'— A4. Then we also 
have a;;=a;;. It is symmetric about the principal diagonal. 


L € 85] 
4 2 8]| =a symmetric matriz . 
5 8-3 
If a;j7 —a;;, so that the signs above the principal diagonal are opposite 
to the signs below the diagonal, then the matrix is said to be skew symmetric. 
A matrix A is skew symmetric if 4'— — A. 
0 —3 4 
8 0 —5 || =a skew symmetric matrix . 
—4 5 0 


If all the principal minors of a matrix are greater than or equal to zero, 
then the matrix is said to be positive-definite. If, in addition, it is symmetric, 
it is a Gramian matrix. 


2 3-3 

2 4 2 || =a positive-definite matrix. 
3 5 6 

5 10 13 

10 20 26 | =a Gramian matriz. 
13 26 36 


In some problems it is important to know the highest order of the non- 
vanishing minors. The highest order of the non-vanishing minors is called 
the rank of a matrix. The rank of Table 5 is equal to its order, namely, 3, 
because the determinant itself does not vanish. The determinant of Table 10 
does vanish, so that its rank must be less than 3. It contains second-order 
minors that do not vanish, and the rank of the determinant is therefore 2. 


Table 10 

10 В 1 
8 8 2| =0 
1 2b 
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If the determinant of a matrix is zero, the matrix is said to be singular. 
If the determinant does not vanish, the matrix is said to be non-singular. 


If [а;;| =0, then aj; is singular. 

If [a;;| 0, then a;; is non-singular, 

Most of the theorems in the elementary theory of real determinants are 
concerned with the methods of ascertaining the numerical value of a de- 


terminant and with the operations that do, or do not, affect its numerical 
value. The following theorems are useful in dealing with determinants: 


1) The value of a determinant is equal to that of its transpose. 


a а а à 

a , 
b, b. а, b: 
а bdb fı dı d» аз 
à; be c | = |b; be bs 
аз bs сз с Ca Сз 


- 2) If any pair of parallel arrays of a determinant are interchanged, the 
absolute value of the determinant remains unaltered but the sign is re- 
versed. 


aq һ б о в с 
+ | @ b е = - |а bh a 
аз b3 сз аз Ӧз сз 


3) If two parallel arrays of a determinant are proportional, the determi- 
nant vanishes. 


a, ka 


аз kaz 
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If k=1, then two arrays are identical and the determinant vanishes. 
4) If a determinant has an array of ciphers, it is equal to zero. 
GB E 
0 о SOS 
def 


5) If each element of an array is multiplied by 


any factor, the value of 
the determinant is multiplied by that factor. 


b bo Us| =k] br b bs 
keı Коз kes | C Co C3 


This theorem is sometimes useful in reducing a 


forms. 

6 98 114 1 Ww 

12 18 4| 2 60x0x2|2 2 2| =6x9xK2x2 1 1l 1 = — 648. 
24 27 2 dud & S8: 1j 


6) If each element in an array is reversed in sign, the value of the de- 
terminant is reversed in sign. 


a b a —aı hi 


e = b, [m 
о be ce} = — | -a b c | = + | +a ياس‎ —% 
аз bs сз —аз b3 сз —@ bs єз 


7) If an array contains no zero elements, all the elements of the array 
may be made unity by means of multiplying factors. 


3.46 he per pude 
28 8|-23x4x6|1 2 {| =3x4|4 12 8 
8 79 Т 2 a $ 

Ше t Le ЛЕ 


3х4х4 |1 3 2| =8х4|1 3 2| = —36. 
7 6 
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8) Every determinant сап be expressed as a sum of two determinants 


(a+p) Dı a [m bh а pha 
| 

(0+0) be a| = | а Ba: dl be Ca 

(a+r) bs ез аз bs Cs # xy £3 


9) The value of a determinant remains unaltered if each element in any 
array is augmented by a multiple of the corresponding element ina parallel 


array. 


(a4 -kb)) bi с | a Dı Cı | Dı Dı cı | a һ @ 


(azt kba) bz ca | = | Q2 be Co | + ‘| ba bz с is аз Ь © 


bs bs Cs | аз b3 Cs 


(as--kb3) bs єз аз b3 ©з 


m р : А А 
The second determinant in the right member vanishes because two columns 


are identical. 


a determinant on one side of the principal diag- 


10) If all the elements of 
ant reduces to the leading term. 


onal are ciphers, the determin 


512 512 3 1-2 NUM ew 
639 |= 633] = 232) = 0 3 3 |= 2х3х5= +30. 
1138 505 005 005 
Matrix multiplication 
Consider the three simultaneous equations. 
Yı + 2% = т, 


Зув P Из = 


(2m + 2y + 5 


(2) 


і 
ә 


are written in the expanded notation. If the x's are known 
be solved numerically for the y's. As an example let 

: : 2 , 
lving the equations simultaneously, 


The equations (2) 
the equations may 
ту=1,%=9, тз=2. 80 
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In some problems it is convenient, as well as clarifying, to represent a set 
of simultaneous equations in the rectangular notation which is illustrated in 
Table 11. Here the coefficients of the three simultaneous equations have 


Table 11 


k k 
|: 2 0| pas f TT 
| 5 | X | =; | г. | 


| 

| x: | y: 
l2 2 x li ys || 
A y 


been arranged in the form of a matrix that may be denoted A. The y's have 
been arranged in a vertical column in a matrix denoted y. The x's are also 
arranged in a vertical column matrix denoted л. The matrix A is of order 
3X3, while the matrices z and y are both of order 3X1. 

A matrix that consists of a single column will be called a column vector. 
A matrix that consists of a single row will be called a row vector. The vec- 
torial terminology is probably due to the fact that the elements in any array 
may be regarded as the Cartesian co-ordinates of a point in a space of as 
many dimensions as there are elements in the array. This point, together 
witb the origin, determines a direction in space. In this manner any array 
of a matrix can be given a vectorial interpretation. 

The three matrices A, y, x, of Table 11 may be regarded as symbolizing 
the simultaneous equations (2). This necessitates that a particular opera- 
tion be implied by the adjacent matrices A and y. In order that these ma- 
trices shall symbolize the simultaneous equations, the following rule must 
be implied in Table 11: 

The first equation in (2) ean be produced from the matrices by writing 


3 
‘ А 
(3) апуп + ашу» + 43/3 = ^N aya = Xn. 
-1 


ss 


In performing this operation with the matrices, a row of the first one is 
associated with a column of the second one. The first equation of (2) calls 
for the cross products of the corresponding elements of the first row of A and 
the first column of y. (In the present problem, 
cross product is illustrated by (9); 

The second equation (2) is produced by performing the same row-by- 


column multiplication, using the second row of A and the first column of y. 
Then we have 


y has only one column.) The 


3 
E 
(4) алуп + 02298 + 23431 = N ао ул = Ya. 
=1 


ч 


— 
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The third equation (2) is produced by a similar operation on the third 
row of A and the first column of y. The sum of the three produets is record- 
ed in the third row and first column of x. The equation is 


5 F LN = 
H A31} A322; A33} аз; Pzr 
(5) зї + as 330/31 - sit 31 


3 
j= 


The three equations may be written in the more condensed form 


3 
NS 
— 


j=1 


а уж = Vik. 


(6) 


This interpretation of two adjacent matrices js called matrix multiplication. 
In the present problem k=1, because y has only one column. Since į сап 
take three different values, namely, 7=1, 2, 3, the equation (6) represents 
all three of the simultaneous equations in swmmational notation. Table 11 
represents the rectangular notation. The three equations (2) may also be 
represented conveniently in the still more condensed matrix notation, 


namely, 
(7) Ау=т, 


which is a matrix equation. The operation specified by this matrix equation 
is that if the matrix A is multiplied by the matrix y, row-by-column, the 
matrix product is another matrix, namely, x. This is an exceedingly power- 
ful method of handling sets of equations, because many otherwise tedious 
numerical operations can be shunted, so that the calculations are performed 
only on a final set of matrices rather than on many intermediate steps. Still 
more important is the fact that significant relations in a problem are con- 
spicuous in the matrix notation but they may be obscure when the prob- 
lem is handled in expanded algebraic or numerical form. 


Table 12 
j i i 
кол Ja 1з EUAN y uu d 
y 3:583 a 
Ont 4 
A’ 


The same set of simultaneous equations (2) may be represented by the 
matrix multiplication shown in Table 12. The multiplication of the first row 
of y' (y has only one row) and the first column of A’ reproduces the first 
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equation of (2). It should be noted that the matrices of Table 12 are the 
transposes of the matrices of Table 11. The transpose of a column vector is 
a row vector with the same elements. 

The summational notation for the matrix multiplication of Table 12 is 


(8) ТУ иза, = Xu. 


The general element of A in Table 11 is a;;. Hence the general element of A’ 
in Table 12 is аз. The general element of y is yi, so that the general ele- 
ment of y’ is у. The matrix equation for Table 12 is 


(9) JA =e 


which represents the same set of equations as (7). 

In order to multiply one matrix by another, the number of columns of 
the first one must be the same as the number of rows of the second. The col- 
umns of A in Table 11 are represented by the subscript 7, and this is also the 
subscript for the rows of y. If the subscripts for the first matrix are 7 and j 
and the subscripts for the second matrix are j and k, then the j subscript is 
eliminated from the matrix product which has the subscripts 7 and k. The 
same rule can be verified in the matrix multiplication of Table 12, where the 
subscripts of the first matrix are k and j and those of the second matrix 
are j and 4. Eliminating the middle subscript j, which is common, the ma- 
trix product has the subscripts k and i. 

The matrix equations (7) and (9) illustrate the following matrix theorem: 

Theorem: The transpose of any product of matrices is the product of their 

transposes in reverse order. 
Hence, if АВ = С, it follows that B’A’=C’. Applying this theorem to the 
present example, we have, by Table 11, Ay=x, and, by the theorem, yA’ 
=x’, which is the matrix equation for Table 12. 

If the z's are known in (2), then the y’s may be found. Let the y’s be 
expressed as linear functions of the z's in (10). 


(10) 22 25 = р, 
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This set of simultaneous equations is represented in Table 18. If the three 


Table 13 


A a yi 
Q 2 . 2 = | 7 
E | 23 уз 

В 2 y 


matrices of Table 13 are denoted В, z, and y, we can represent the three 
equations (10) in the single matrix equation, 


(11) Вг = у. 


Since the y's are known, the values of the z's can be determined. Substitut- 
ing the known values of the y's in (10), we find that 


a= 115 giu; Lus 
Equation (7) shows that the z's can be expressed linearly in terms of 
the y's. Equation (11) shows that the y’s can be expressed linearly in terms 
of the z's. It is desired now to express the z's directly in terms of the z's 
without the intermediate y’s. This сап be done. From the equations 


(7) Ay = =, 


(11) Вг = у, 


it follows that 


A(Bz) = z, 
ABz =a 

Let AB=C. Then 
(12) Cz=2. 


In order to express the z's in terms of the z's, the matrix product AB=C 
must be determined numerically. This matrix product is shown graphically 
in Table 14. Consider the first row of A and the first column of B. The cross 
product is 


(1)(1) + (2)(0) + (001) = +1. 
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This is therefore the element in the first row and the first column of the 


matrix product C. Consider, as another example, the second row of A and 
the third column of В. The cross produet is 


(0)(1) + )1)+ (1)(0) = +3. 
This is the element соз in the matrix C. 


Table 14 


a 


ЛЧ Л =] d 8 | 
2 €] 1 5:0] 16 12 4 
A LB C 


Since the numerical values of the z's, the y's, and the z's are known, the 
matrix equation (12) may be tested graphically, as shown in Table 15, Asa 


Table 15 


1 ка —51! 1 
[D 864 RESET 
he wal l-al jal 

C z ž 


sample check, consider the second row of C and the first column of 2. It 
should reproduce the value 191 =3. 


(00-0) + (06) + (780 = +3. 


Table 14 shows the matrix product AB=C. If the order of the matrices 
A and B is interchanged in this multiplication, a different product is ob- 
tained. This is readily verified numerically in Table 14; and it illustrates the 
principle that if AB=C, then, in general, BAC. Matrix multiplication is 
not commutative. In matrix algebra it is essential to note the order of the 
matrix factors because the order is not arbitrary, as in ordinary algebra, 
where ab = ba. 

The following is an example of matrix algebra. If, instead of (7) and (11), 
the transposed forms of these equations were used, we should have 


(9) УА — a' 


(13) zB a. 
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Substituting (13) in (9), we have 


(14) SEA =. 
But. 

(15) АВ = С. 
Hence 

(16) B'A = C. 


Substituting (16) in (14), we obtain 
(17) 216 =, 


which could also be written directly as the transposed form of (12). 

In order that there shall be a unique solution for the simultaneous equa- 
tions (2), the matrix A of the coefficients must be non-singular, i.e., | А | 0. 
This may be tested by trying to solve a set of non-homogeneous simultane- 
ous equations with coefficients whose determinant does vanish. 

The multiplication of matrices is associative. This is illustrated as fol- 
lows: 


(AB)C = A(BC) = ABC. 


The matrix product (1B) may be determined and then postmultiplied by 
C, or the matrix product (BC) may be determined and then premultiplied 
by 1. The product is the same. This principle can be extended to any 
number of matrix factors. For example, à 


(ABC)D = (AB)(CD) = A(BCD) = ABCD. 


Note that the order of the matrix factors is retained. 
The sum, or difference, of two т Хп matrices is the n Xn matrix each of 
whose elements is the sum, or difference, of the corresponding elements in 


the given matrices. 


ll 
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The components may be written in any order. 
A+B=B+4+4; 
(A+ B)4+C=A4(B4+C)SA4 B40. 
If k and m are scalars, then 
kA + kB = k(A + B); 
kA + mA = (k + mA. 
The multiplication of matrices is distributive. 


A(B + C) = AB + AC; 
(B+ C)A = BA + СА. 


Ш 


It is sometimes useful to know that the determinant of the product of two 


square matrices is equal to the product of their determinants. The following is 
an example: 


2 6 


2 1 
Let |A| — = —10,  andlet |B) = =4+5 
4 3 4 
Then 
26 
|A] + |B] 2 |AB| = = — 50 = (—10)(4-5). 
29 32 


In the matrix product AB, the matrix B is said to be premultiplied by the 
matrix A, or the matrix A is said to be postmultiplied by the matrix В. 

The operation of multiplying one matrix by another can be summarized 
for mnemonic purposes in the diagram of Figure 1. This diagram shows 
that rows of the first matrix are associated with columns of the second 
matrix and that the middle subscript is eliminated in the produet. If the 
ith row of A is cross multiplied with the kth column of 
is recorded in the cell ik of C. 

There is nothing magical or profound in the p 
multiplication that have become conventional. 


B, the cross product 


artieular rules of matrix 
The row-by-column rule is 


m 
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entirely arbitrary. It would have been possible to set up a column-by-row 
rule provided that the matrices had been so arranged that the rule would 
have reproduced the original equations which the matrix notation represent- 


k 


A AB=C 


Ficure 1 


ed. It would also have been possible to have a notation which implied that 
one matrix was on top of another, but this would not have been so conven- 
ient for writing habits that go from left to right. 


Diagonal matrices 

In the manipulation of systems of equations there occurs frequently a 
type of matrix in which all of the elements are zero except the diagonal ele- 
ments. A matrix in which only the elements of the principal diagonal are 
non-vanishing is ealled a diagonal matrix. The following is a diagonal ma- 


trix of order 4: 8.C.E.R.T., West Benga) 
a 000 Date... 3,98 —_— 
i. oae 


0b 0 0 

= D = a diagonal matrix. 
00 € 0 
000d 


It sometimes happens that all of the elements of a diagonal matrix are 
identical. Such a matrix is ealled a scalar matrix. The following is an ex- 


ample: 


k 0 0 0 
Û k& 0 0 " | " 
= K = a scalar matrix. TEN 
0 ffs 
ЖЕ Arr 
000 ЕЁ ks 5 
+ К 
p 
>. Calcutta 
Accession МӨ. «««овевогов — ? 


2 
Ww 
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When a diagonal matrix has unity in each diagonal cell, it is called a 
unit matrix or the identity matriz. The following is an identity matrix of 
order 4: 


Ji ooo 
li 
отоо 
= І = the identity matrix. 
00 10 
00 0 1| 


The properties of diagonal matrices are very useful in handling sets of 
linear equations: 

1) Premultiplication DA with a diagonal matrix D multiplies each row 
of A by the corresponding element in D. 

2) Postmultiplication AD with a diagonal matrix D multiplies each 
column of A by the corresponding clement in D, 


АШИ 


› 


Ш 


0 q q dq 
D DA 

a b р 0 ар bq 

c d 0 q 5 cp dq 
A D AD 


3) Premultiplieation or postmultiplieation with a scalar matrix А mul- 
tiplies all elements of A by the constant element of K. This is a special case 
of the first two theorems. "The reason why premultiplieation with a scalar 
matrix has the same effect as postmultiplication is that if every row is mul- 
tiplied by a eonstant p, the effect is the same as if every column is multiplied 
by the constant p. In either ease every element of A is multiplied by p. 


a b p 0 ap bp 
c d 0 p cp dp 
A K AK 
p 0 a b ap bp 
0 p c d cp dp ' 
K A KA 
p 0 a b a b 
-p А 
0 p c d c d 
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Since the effect of a scalar matrix is independent of its position before or 
after the other matrices in a matrix product, its constant element p can be 
used in the product instead of the scalar matrix A as shown in the third ex- 
ample. This illustrates the following theorem: 

4) If K isa sealar matrix, then its constant element p may be substituted 
for the sealar matrix in a product. 


AK = КА = pA = Ар. 


A multiplier which is independent of the non-commutative rule of matrix 
algebra is called a scalar. 

5) To multiply a matrix A by a scalar p, in either order, pA or Ap, is to 
multiply each element of 4 by p. 

The identity matrix is a special case of the scalar matrix, and hence it is 
also independent of the non-commutative rule of matrix multiplication. 


| 1 0 a b a b 
| 0 ded ie afl’ 
I A IA-A 
a b E a b 
c d "Joi c dl 
A I AI=A 


6) To multiply a matrix A by the identity matrix, in either order, 47 or 
TA, is to reproduce the matrix A unaltered. 


АІ = ІА = А. 
The identity matrix Z in matrix algebra corresponds to unity in ordinary 
algebra. Hence the identity matrix is suppressed, just as unity is suppressed 


in ordinary algebra. 


1x5=5, 
1ХхХх=х, 
IA = A. 


The inverse 
If a matrix A is non-singular, i.e., |A|#0, then there exists another 


unique matrix such that its multiplication by A produces the identity ma- 
trix. This other matrix is called the inverse of A, and it is denoted A-! 


Hence, if A is non-singular, 
add" є [ 
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The inverse of A~ is A, so that 
АА = І = AA. 
Consider the ordinary algebraic equation, 
ах = у. 


If it is desired to state x explicitly, the equation is ordinarily written ах 


If the given matrix equation is 
AB ed 


and if it is desired to write it explicitly for B, this cannot be accomplished 
by ordinary division. A matrix is not a number but a rectangular table of 
numbers, There is an operation in matrix algebra which corresponds to 
division in ordinary algebra. If both members of the matrix equation AB 
=C are premultiplied by A, we have 


AAB = Aig 
But 
AMA = Т, 
Hence 
IB = AC, 
or 


B= AC, 


This is the desired form. This example illustrates the operation in matrix 
algebra which corresponds to division in ordinary algebra. It consists in 
moving a premultiplying or postmultiplying factor from one member of the 
equation to the other member in the same relative position. This operation 
is illustrated in the following examples: 

If 
ABC = М, 


then 
BC = AM, 


С = BAL, 
CM“ = В-\А-\, 
CM-A = Bo, 
MA = capa 


25 
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Since this operation is analogous to ordinary division, the inverse of A is 
sometimes called the reciprocal of A. 

The inverse of any product of matrices is the product of their inverses in 
reverse order. 


Let ABC=M, 
BC = AM, 
C= BAM, 
I = C3 B2A2M , 
M-' = CB-'A-" , 


But 
(ABC)? = М. 


Hence 
(ABC)? = COBOA-, 


A method of writing the inverse of a given matrix is as follows: Let the 
given matrix be A with elements aij. 
1) Write the matrix M with elements m:; which are the first minors of 


the elements аў; | 
2) Reverse the signs of alternate elements of M so that it becomes the 


matrix Z with elements eg = (—1)'*mi: 
3) Write the transpose of E, namely, E' =F, with elements fi, =e; The 


matrix F is the adjoint of A; | 
4) Divide each element of F by the value of the determinant |A|. This 


is the inverse A7 with elements 


= fü 
jn = lAl* 


The writing of an inverse will be illustrated by the numerieal example of 
: ы H H H 
equations (2). The given matrix equation is 
Ay =a. 
It is desired to find the inverse of A so that the equation 


у = Ae 


may be written in numerical form. 
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] 4-9 0 


oc 
| 
to 
Ш 
— 
= 


2 4-1] =P 


— 6 2 3 
o Smock 
14 14 11 
= 4-1 14-71 1 
mt ip тэл =A [А 987 
[ 2 
Ti 14 14 


It is of interest to verify numerically the matrix equation у= Aa, It is 
written in rectangular notation in Table 16. 


Table 16 


1a ya ri 1 _ц 

“ “ ¬٠ з= 11 
ا سے‎ 2 4 9 6 

n" ra Ya 2 "à 
Act x y 


In a later section of this Mathematical Introduction will be found a prac- 
tical computing method for the inverse. The method described above is not 
suitable for matrices of large order. 

The characteristic equation 

The characteristic equation is of considerable theoretical interest in fac- 
tor analysis, and it appears in several of the fundamental faetor problems. 
For this reason it is deseribed in this introduction. In a more complete 
didactic presentation of this subject, the characteristic equation should be 
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introduced with some geometric and other interpretation, so that the sig- 
nificance of this equation might be apparent. The relation of the charac- 
teristic equation to the problems of factor theory will appear in later 
chapters. 

If a constant 8 is added explicitly to each diagonal element of a square 
matrix A, the resulting matrix is called the characteristic matrix of A. It is 
illustrated as follows: 


ап аз аз 100 (ап +8) ау» аз 
Qn Qz (es В 01.0 = аз (аз-Е8) a23 
аз\ A3 A33 001 аз аз (aa34- 8) 


A ВІ „ (A+B) 


Characteristic matrix of А. 


The determinant of the characteristic matrix is the characteristic determi- 
nant of A. 

The expansion of a characteristic determinant of order r is a polynomial 
of degree r. When this polynomial is set equal to zero, the equation is called 
the characteristic equation of A. An example is the following equation: 


(1+8) 0 1 
р 1 Q8 1 |=о. 
1 0 (2+8) 


When the determinant is expanded, the characteristic equation becomes 
@ + 58° + 78 +2 = 0. 


fficients of the expansion of a characteristic determinant can be 


The coe 1 : х : 
f the prineipal minors without expanding the whole de- 


written in terms o ipa s 
terminant. Let the characteristic equation be as follows: 


(18) вела T agr eps e onem 0s 

Then the coefficient mz is the sum of all the z-rowed principal minors in А. 

The coefficient m; is the sum of all the 1-rowed principal minors of A. These 

are 1, 2, 2, and the sum is +5. The coefficient ms is the sum of all the 
? 3 Lu 


éj ^ 
2-rowed principal minors of A. These are 


1 0 


1 2 
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The coefficient m.=2+1+4=+7. The coefficient ms is the sum of all prin- 
cipal minors of order 3. This is the determinant |4 | itself, and its value is 
+2. The coefficient of the highest power of 8 is unity. These coefficients 
can be verified by expanding the determinant. 

Another method for finding the coefficients of a characteristic equation 
has been described by Paul Horst in the Annals of Mathematical Statistics, 
Vol. VI, No. 2 (June, 1935). 


The summational notation 


If a set of n numbers is to be summed, the operation may be indicated in 
the erpanded form, 


(19) wy SF tee а 4-42. 


The same operation may be indicated in the more condensed summational 
form, namely, 


(20) Soni =, 


in which the subscript i takes the successive integral values from 1 to n, 
inclusive. 

In statistical work it is pedantic to indicate the limits, because the sum- 
mation is over the entire population exeept in rare cases, which can be 
specially indicated. It is acceptable practice in statistical work to write Sx 
without subscripts when the usual form of summation over the population 
is implied. In factor analysis this simple and convenient notation becomes 
ambiguous because summation may be over the factors, over the popula- 
tion, or over the variables. It is therefore advisable to adopt the unam- 
biguous double-subseript notation that is conventional in mathematies. 


As an example, the sum of the elements in the first row of Table 2 can be 
written in the form 


an + аз + ais +++) +a, = Уа. 
— 


Here it is the second subscript j, representing the columns, which is found 
in the summation sign. This means that the a’s are to be summed for all 
values of j from 1 to n. The first subscript is fixed. The summation is there- 
by confined to one row. 


The notation ean be generalized to represent any row i 


. It then takes 
the form 


аа + aa + aa d d as = Nas. 
— 
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This notation means that the a's are to be summed for a fixed value of 7, 
since i does not occur in the summation. The a's are to be summed in one 
row i for all the column values of j from 1 to n. 

By analogy, the sum of all the elements in a column j of Table 2 may be 


represented as follows: 


m 


а; + а; + йу btt db du; = Уа, 


Here it is the column j that is fixed because it does not occur in the summa- 
tion sign. The a’s are to be summed in some specified column j for all values 
of i from 1 to m. 

If it is desired to designate the sum of all the elements in the matrix, each 
of the m-row sums must be summed. This involves a summation over both 
i and j. We then have 

n n 


CAU QUON 
= У “ау. 


i=l j=l 


Sum of all elements in А 


Since it does not matter whether the rows are summed before the columns, 


or vice versa, we have 


n m m n 
NERA, ^ E 

У Уа; => > gi. 

ہے ہے a‏ — 

jal i=1 j=l 


also be designated by the summational no- 


A matrix multiplication can 
т, бев | ation АВ=С of Table 17. 


tation. Consider the matrix multiplic 


Table 17 


ўт) k(p) kip) 
Cun Cis Cis 
2 (13 
ds m bu bu bis 
Ca Cn Co 
ы a. 823 ш Е 
ад 02 йл) ba ba Das =i(m) 
i(m) P Cn Ca Cos 
p a33 
ап ап " | ba ba bss 
Ca Ca C 
dus dues deis 4 Са ^a 
1 B AB=( 


o matrix of order m Xn with general element ai; Let B be a 
ай v pecie s general element bite Then the product A B=C 
iet [p^ à matrix of order mxp with the general с Ci. The middle 
subscript j for the general element disappears 1n We product, and so does 
the middle dimension п in the product (тп) (пр) = (np). In the example, 


m=4, n=3, p=3. 
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The element cn is obtained by the cross multiplication of the first row of 


A and the first column of B. In suminational notation, 


, 


A" 
(21) anba + anba + agba = Mab = CH. 
1 


А4 


The second row of A and the first column of B: 


п 
A 

(22) азба + aba + aba = азд = сл. 
j=l 


The ith row of A and the first column of B: 


n 


(23) andy + аә + aiba = “афа = Ci. 
j=l 

The ith row of A and the kth column of B: 

(24) aab + аш + aigba: = Sadi = Ci 
ј=1 


The summation of (24) gives a single element in C. If it is desired to indi- 
cate the sum of all the elements in the ith row of C, the summation will be 
over k with a fixed value for i. We have then 


(25) NN NC 
— 


aub; = 


k=l j=l Е=1 


Finally, if the sum of all the elements in C is to be indieated, the summation 
must also be made for all rows. Then 


m p m m p 
ее: AN 

(26) ^ > абу: =V Cik 
t=1 k=1 j=1 i-1 k=1 


The summation of (24) represents the multiplication of two matrices, aij 
and bj, whose product is the matrix са. It can be visualized in Figure 1. 


MATHEMATICAL INTRODUCTION 31 


As an example of manipulation with the summational notation, consider 
n r + . J i 
the first term of equation (12-i). It is 
¥ r 
lS NS gas 
7 аула? у 


N— 


i=] т=1 


in which it is desired to substitute 


in order to simplify the first term. 
Since the order of summation is 
may be interchanged. Then the first term becomes 


arbitrary, the order of the summations 


r N 
ISSN а 
№24 2 Unni + 
m=1 i-l 


occur in a}, this factor is a constant during 


Since the subscript 7 does not 
it may be placed in front of the summation 


the summation over i. Hence, 1 
over i without altering the value of the first term, whieh then becomes 


r N 
ins ag 
> a, > Umi ° 
М = ہے‎ 
m=1 i=l 


and so it can be placed anywhere in the 


cal of N isa scalar, 
the first term becomes 


But the recipro ls 
ing its relative position, 


summation. Chang 


ade more clearly. Suppressing the part 


stitution can be m 
simplifies into 


the expression 


* 
2 
Gm + 
m 


and now the sub 
which is equal to unity, 


will ordinarily be found necessary, but 
У, 


ore explicit than 
in which the summational notation can 
В 


These steps are m 
her the manner 


they illustrate furt 
be handled. 
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Linear dependence 


A matrix of order mXn may be regarded as m sets of numbers with n 
numbers in each set. Each row of numbers is then a set. Table 18 is a matrix 


Table 18 


| & 6 1) 9. а 
pod E 204 
| 5 m 15919 8 
of order 4X6. In this table every row сап be expressed linearly in terms of 


the first row. By this is meant that for any row 2 there exists 


а constant Ci 
such that 


(27) ап = Cil. 
For the fourth row the constant с; is 3, so that each element in the fourth 


row is three times the corresponding clement in the first row, When any 
row can be so expr 


ssed in terms of the first row, the rows are proportional, 
and it can be shown that the columns are then a 
rows are not proportional, they 
of them 


lso proportional. If two 
are said to be linearly independent, for one 
cannot be expressed linearly in terms of the other. When each row 
can be expressed linearly in terms of one row, the rank of the matri: 
This means that all second-order minors vanish. This fact is readily verified 
in Table 18. 


The ide: 


is 1. 


1 of proportionality can be generalized to two or more dimensions. 
An example of rank 2 is shown in Table 19. In this matrix any row can be 
Table 19 


P5464 BF 


to 


22) £ * 4 
© 45 65 4 
97 4 1) Ww 6 


expressed as a linear function of 
there are no two dependent rows 
€» (not both zero) exist such that 


any two rows. In this particular matrix 
- This requires that two constants c; and 


(28) аң = C0, + Cos; , 


where the elements in the ith 


TOW are expressed linearly in terms of the 
first two rows. 
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In the following example the two constants for the third row of Table 19 
are determined. For the first two entries in the third row, 
| 5 = Та + 20. 
(29) 
| 4 = ба + 20. 


Solving (29) simultaneously, we find that c; =1/2 and c» —3/4. Testing this 
on the last column, as an example, 


(2) + (00 = 4. 


A different set of constants must be determined for each successive inde- 


pendent row. 

Since each of the rows in Table 19 can be expressed linearly in terms of 
the first two rows, it can be shown that the matrix of Table 19 is of rank 2. 
This implies that all third-order minors vanish. As an example, the fol- 


lowing third-order minor of Table 19 vanishes. 


D. x8 of 
4 6 5] =0. 
7 10 9 


It can be shown that if the rank of a matrix is 7, then there exists a set 

of r columns, or rows, in terms of which each column, or row, ean be linearly 
ing! J “=, 

expressed. 

on of linear equations 


Geometric interpretati 5 ee Е 
a straight line in a plane is prob- 


The most frequent form of equation for 
ably 


(30) у = тх FP, 


: MERE ;hile m and p are two independent 
BRE re the two variables w Р 

S y a К e SrA with the well-known fact that any two points de- 
ia 2 i ht line. The multiplying constant m is the slope, and the 
, m as oui » is the y-intercept. In the present context it will be more 
de ae nein pi the equation of a straight line in the more general form: 

seful to b 
(31) ат + айз + Ё 70. 


В " r and three parameters, ai, as, k 

2 E ‚о variables, 21 and 2, a 6 par ‚а, аз, k. 

д үн а клЫА êêê to determine the line, it follows that the 
Since only two points ? 


T dent. 
three parameters are not indepen 
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Equation (31) can evidently be multiplied by any arbitrary constant 


without affecting its geometrical representation in the plane. Then (31) 
becomes 


(32) caıT + сахо + ck = 0. 


Let the multiplier c be so chosen that the sum of the squares of the coefficients 
of the variables is equal to unity. Then 


(33) (cai)? + (саз)? = 1, 
or 
(34) (a 


af @ 


Let ca; = №; саз X; ch=d. Then 


(35) ма + Ммъ+4=0, 
where 
(36) M+N=1. 


When the equation is written with 


this adjustment, it is said to be in 
normal form. 


This definition is applied not only to the equation of a line 
In a plane, and to the equation of a plane in a space of three dimensions, 
but also to the equation of a hyperplane of (n — 1) dimensions in a space of 
n dimensions. The number of dimensions of the space defined by equation 
(35) is (n — 1) where n is the number of variables. Hence equation (35) de- 
fines a space of one dimension, a line, in a space of n 22 dimensions, a plane. 
When a linear equation is in the normal form, the parameters have in- 
teresting meaning. The parameters ^, and № are the direction cosines of the 
normal to the linear space which is defined by equation (35); and the 
parameter d is the distance from the origin to the same linear space. The 
normal to the line makes cos-!A, with the a-axis and cos“), with the tz- 
axis. The direction cosines of a space are the cosines of the angles that its normal 
makes with the Cartesian co-ordinate axes. In the present case the space is à 
one-dimensional space, namely, that which is defined by equation (35). In 
order to avoid ambiguity, the normal is taken positive on the side which 
contains the origin. Equation (35) may be interpreted geometrically as 
defining a space of one dimension whose normal has the direction cosines 
Мм and № and which is distant d from the origin. 
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If the parameter d vanishes, then the space which is defined by the equa- 
tion contains the origin. In equation (35) the line contains the origin if d 
is zero. 

Equation (35) locates a one-dimensional space (a line) in a two-dimen- 
sional space (a plane). If a new variable, тз, is added, the equation takes 


the form 


(37) Aug Mrs + Mrs dd = 0. 


It defines a space of two dimensions (a plane) in a space of three dimensions 
with three orthogonal axes. Here, as before, if the equation is in normal 
form, then d is the distance of the plane from the origin, and the three coeffi- 
cients М, №, Аз, are the direction cosines of the normal to the plane. They are 
the cosines of the angles that the normal makes with intersecting lines that 
are parallel to the ti, 22, and аз axes, respectively. 

‘The direction cosines have the property that the sum of their squares is 
е line is defined if the parameter d and one of the 
In equation (37) the plane is defined by its dis- 
tance from the origin and any two of its direction cosines. The third direc- 
tion cosine ean be found from the fact that the sum of the squares of the 
direction cosines equals unity. If d vanishes in (37), the plane contains the 
origin. A plane through the origin is therefore defined by its direction co- 


sines, which are the direction cosines of its normal. 
An equation of the same form in n variables is 


unity. In equation (35) th 
direction cosines are given. 


(38) мл + № oe Nate ТЕ ss + мє Fd =0. 

1 —1) dimensions in a space of » dimensions. If 
stance of the hyperplane from the origin. 
f the normal to the hyperplane and hence 


It defines a hyperplane of ( | 
(38) is in normal form, dis the di 
The A's are the direction cosines O 


n 


(39) 24-1 


i=l 


In the present factor analysis the hyperplanes of primary interest contain 
емак i Tl 
‚ d vanishes. Then 
the origin, so that the parameter d ушы 


(40) ма + Мм + dats Tee + Anan = 0. 


The | of A, are said to be the direction eosines of the hyperplane L 
n values of ^i 2 ds 


which is defined by its normal A. 
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A matrix may be given a geometric interpretation. Let the matrix A be 
of order mXn. Then the n elements of each row may be regarded as the 
Cartesian co-ordinates of a point in n dimensions. Since there are m rows, 
the matrix may be thought of as defining the positions of m points, one for 
each row, in a space of n dimensions. Table 20 is of order 6X3. It can there- 
fore be regarded as defining the positions of six points in a space of three 
dimensions. 


Table 20 


1 2-4 
—4 1 -1 
-2 -3 5 

3 5 -9 

4 —2 14 

ر 0 


Let the rank of an mXn matrix A be r. Then it can be shown that the m 
points are contained in a Space of r dimensions which also contains the 
origin. The rank of the matrix of Table 20 is 2. Hence the six points should 
lie in a plane which contains the origin. The equation of such a plane is 


(41) Mit + Now, Mr = 0, 


in which the z's are the three co-ordin 


ates of each of the points in the plane 
and the № are the direction cosi 


nes of the plane. The № are not independ- 
ent parameters because of the conditional equation (39). Hence any two № 


define the plane. These may be found by any two of the six points which are 
not collinear with the origin. Since no two rows of Table 20 are propor- 


tional, no two of the points are collinear with the origin. If two such points 
were found, then these two points would define a line through the origin and 
not a plane. 


Substituting the z's of the first two points of Table 20 in (41), 


(42) M+ 2% — 4, = 0 


, 


k= ONE м Ду. 


Solving for А, and X in terms of Аз, we have 


(43) | ==, 


№ = ЗА. 
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Substituting (43) in (89), 


м) 4 + 9М +۸3 =1, 
Or 
vi 


and hence 


2 

AET EN 
714 

TT 
vM 


2 d 1 
15 e perd Xe + te = 0). 
ч) T s Far iti 


points must lie in this plane, since the rank of 


All of the four remaining UHR à 
s are the direction cosines of the plane, 


Table 20 is 2. The three coefficient 
tion cosines of the normal to the plane. 


The distance of a point from the origin is 1 Ez?, where the 2’s are its 
co-ordinates. For example, the distance of the fourth point from the origin 
is 1 32-L52-+(—9)?= 10.72. If the sum of the squares of the co-ordinates of 
a point is equal to unity, then the point is at unit distance from the origin. 

Each point may be interpreted as defining the terminus ofa vector from 
the origin. The scalar product of any two vectors is hha gos Ф, where A; and 
he are the lengths of the vectors and ¢ is their angular separation. If the two 
vectors are of unit length, then the scalar produet is the coing of Be angular 
Separation. It ean be shown that the scalar product of two veetors ean be 


expressed in the form 


i.c., the direc 


" 
ы 

Novit, 

— 

j=l 


а (rows) and j refers to co-ordinates (columns). 
"here ? ofer to points (rows) а 
up , a с би product of the vectors defined by the second and 
‘or example, » seals 


fourth points of Table 20 is 


(—4)(3) + (1)(5) + (-11(-9) = + 92. 
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If the sum of the squares of the co-ordinates of each point is equal to unity, 

so that the points lie at unit distance from the origin, then the scalar prod- 

uct, or cross product, is the cosine of the angular separation of the vectors 
5 

at the origin. 


Orthogonal transformations 


The three simultaneous equations (2) may be regarded as representing 


the three co-ordinates of a point x (zi, тз, £3), the three co-ordinates of a 


point y (Yı, ys, ys), and the law by which each point y is transformed into a 
corresponding point z. For every point Y, there exists some other point x 
whose co-ordinates can be found by (2) when the co-ordinates of y are 
known. This relation is called a linear transformation by which the points y 
are moved to the corresponding points z. Every pair of the corresponding 
points is related by the linear transformation (2). The transformation is 
called linear when the equations by which the a's сап be found from the 
y's are of the first degree, as is the case in (2). 

If the transformation is of such nature th 
from the y's by merely rotating the co-ordin 
tion is an orthogonal tran 
that a transformation sh 


at the zs сап be obtained 
ate axes, then the transforma- 
sformation or rotational transformation. In order 
all be orthogonal, it is evidently necessary that the 
275 be at the same distance from the origin as the corresponding y’s because 
the distance of a point from the origin remains invariant when the co-ordi- 
nate axes are rotated. It is also necessary that the angular separations, or 
scalar products, be invariant, because the configuration of the points is not 
altered by rotating the co-ordinate axes. 

The matrix A of Table 11 is called the matrix of the transformation when it 
is regarded as the relation by which the points y are changed into the 
points z. A linear transformation is represented in the more general form 
by the square matrix of Table 21. It can be shown that à square matrix is 


Table 21 


Ain 
аз 0» an dan 
ад аз da аз 
ж, Жука а 
Gn аш аш л. 


orthogonal, i.e., that it h 
of points т, with the s 
conditions: 


as the effect of rotating a set of points y into a set 
ame configuration as the V's, if it satisfies the following 
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1) The sum of the squares of the elements in each row is equal to 


unity, i.e., 


(16) Sef = 1, 
iat 
and 


2) The eross product of every pair of rows, î and |, is equal to zero, i.e., 


(47) N аңа = 0, 
— 
ј=1 
when (0. It is immaterial whether the rotation is conceived as a rotation 
onal co-ordinate axes in a fixed configuration of points or as a 


of the orthog 
a fixed reference frame of the co-ordinate 


rotation of the configuration in 
axes. The result is the same. 
The two conditions, (46) and 
it is sometimes convenient to combi 
be done by writing the conditions in 


(47), are of such frequent occurrence that 
ne them in a single statement. This can 
the more condensed form 


n 


(48) ôu = Sain , 


jel 


where the symbol ё is known as Kronecker's delta. It is defined as follows: 


ба = + 1 when i=l, 
би = 0 when? # 1. 


this definition of би, the single statement (48) 
6) and (47). 


ation is square and if it 
tisfied: 


It can be seen that with 
covers the two statements (4 


If the matrix of a transform: 
are also 5% 


satisfies (48), then the 
following conditions 


а he elements in each column is unity, i.e. 
3) The sum of the squares of t pis 


(49) 
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4) The cross product of any pair of columns is zero, i.c., 


n 


(50) > ашаа 20, j#k, 


i=l 


where j and k refer to columns. ' 
5) The determinant of the transformation is +1, i.e., 


(51) [А] = +1. 


If an orthogonal co-ordinate axis is reversed in direction, then the corre- 
sponding co-ordinate for each point is reversed in sign. If an odd number of 
orthogonal co-ordinate axes are reversed in direction in an orthogonal trans- 
formation, then the determinant of the transformation is equal to —1. If 
an even number of axes are reversed, the determinant is equal to +1. These 
two statements can be made with reference to the configuration. If the ro- 
tational transformation retains the configuration of the points, the determi- 
nant of the transformation is equal to +1. If the rotation involves a sym- 


metric distortion of the configuration, the determinant of the orthogonal 
transformation is equal to —1, 


Table 22 


My 9 15 Y, Yi Y: 

р 9 929 

E a 866 —.500 a | == „1394—3988 

by 2 3 | 500.866 || © 1232 — 1.508 

Ge || sre +4 с 1.134 3.964 

a A 

аб 5 -2 а | 3330 —4.232 
ý n 1 e 3.964 —1.134 [| 

A AG=B 


Each column of an orthogonal transformation 


shows the direction cosines 
of one of the new co-ordinate axe 


5, referred to the given co-ordinate axes. 

A rotation of the co-ordinate axes implies that the given configuration 
and the transformed configuration are contained in the same space. The 
rows of a transformation correspond to the dimensions of the given configu- 
ration, and the columns of a transformation correspond to the dimensions 
of the new configuration. If the transformation is merely a rotation of axes, 
it is evident that the matrix of an orthogonal transformation is necessarily 
square. If the matrix of a transformation is of order mn where туп, 
then the transformation cannot be orthogonal, since the number of dimen- 
sions of the given configuration and the number of dimensions of the trans- 
formed configuration are not the same. However, such a matrix may satisfy 
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condition (47), and it is then said to be orthogonal by rows. If the condition 
is satisfied for columns, as in (50), instead of for rows, then the matrix is 
said to be orthogonal by columns. 

Table 22 is a numerical example of the rotation of five points in a plane. 
A is a matrix of order 5X2. The orthogonal transformation G is of order 
2X2. The matrix product AG — B is of order 5X2, and it shows the co- 


Ficure 2 


s with reference to the new rotated co- 


qu , ive point 
ordinates of the same five P ts have been plotted for the given 


ordinate axes, In Figure 2 us pr rx ENC ur cit 

or g "din: axes, 1 22 v 

i ү. Ят ае пне these axes through an angle ¢=30°. The usual 
«et it be desir "s 


formulae f tation of axes* can be written in the form of a 2X2 transfor- 
ilae for rota ai 


Mation as follows: | 


olid Analytic Geometry (New York: 


cosó —sin Ф 


sin ¢ cos ф 


i Se 
* W. F. Osgood and W. С. Graustein, Plane and 5 


Macmillan Co., 1929), p. 220. 
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The first condition (46) gives 
cos ¢ + sin ¢ = +1, 
and the second condition (47) gives 
cos ф sin ¢ — cos ¢ sin ¢ = 0. 


Hence this is an orthogonal transformation in which the other properties 
may be readily verified. 


FIGURE 3 


Substitution of ¢ = 30° in the transformation produces the matrix G, and 
the multiplication AG produces B. The numerical values in B may be 
checked in Figure 2, where У, and Y» have been drawn so that Y,OX,;=¢ 
=30°. For example, the co-ordinates of the second point can be measured 
on the graph to be 3.23 and 1.60 for the two rotated 


axes, while they are 
2 and 3 for the two original axes. This figure illustr 


ates the geometric in- 
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terpretation of an orthogonal transformation, The two columns of the trans- 
formation G show the direction cosines of the new orthogonal reference 
vectors Y; and J. 


Oblique transformations 

In Figure 2 the two co-ordinate axes Y; and F: are orthogonal. If it is 
desired to define the n points with reference to oblique reference axes, the 
transformation is effected in a similar way. In Figure 3 the same five points 
are plotted on the Ху and X» axes which are implied in the given matrix A 
of Table 22. Here the new axes are oblique; Zi is rotated 30? from Xi, and 
Z is rotated 45° from Xs. 

In Table 23 are shown the numerical values for the corresponding oblique 
transformation. The matrix A contains the co-ordinates of the given five 


Table 23 


BTE Ln Z: Zi Za 
a 1 -2 .S00  —.707 a || = .134 —2.121 
b 2 3 ў | ‚500 .707 | = b 3.232 .707 
с -1 4 с 1.134 3.535 
4 eee H d 3.330 —4.949 
e 4 1 € 3.964 —2.121 
A AH=C 


points in a space defined by the two orthogonal reference axes Хз and Xs. 
The matrix IJ is a square matrix of order 2X2. It is the matrix of the oblique 
transformation. Its columns show the direction cosines of the new oblique 
co-ordinate axes Z, and Zs. These direction cosines may be verified in 
Figure 8. 

It should be noted that the sum of the squares of each column of H is 
equal to unity. Each column of // may be regarded as defining a unit vec- 
tor in a space of two dimensions. The cross product of the columns of 
IT is 


Ni + Мэш, 


which is the cosine of the angle between Zi and 42. 
The product АН = С shows the projection of each of the five points on 
each of the oblique axes Z; and Zs. This interpretation can be verified by 


actual measurement on Figure 3. 
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A method of finding the roots of a polynomial 


Consider a polynomial of the type 


(52) f(8) = cB’ -aB' + eB" cB +e = 0, 
where r is a positive integer, co#0, and Co, Cı, .. . , c, are real coefficients. 


The r roots, 8, of this equation are desired. Determine the upper and lower 
limits* of the roots of this equation. Let a trial value of 8 within these 
limits be В". If 8' is a root of the polynomial f(s), then by the Remainder 
"'heorem,f f(8') 20. The numerical value of f(8’) may be determined on an 
electric calculating machine by computing /(8)/(8— 8”) by the process of 
synthetic division. Consider the sign of the numerical value of f(8’) and 
select a second trial root, designated 8”, which will give f(8") opposite in 
sign to that of f(8’). When two such trial values of 8 are found, there is at 
least one root$ between them. Determine a third trial value, 8’, by linear 
interpolation between f(8') and f(8"). If the value f(6’) =0, then 8% is one 
of the r roots of the polynomial. If f(8"^) #0, interpolate for successive trial 
values until that value of 8 is found for which the remainder, f(8), is zero 
to as many decimals as required. 

Repeat this process for each of the r roots of the polynomial by taking 
trial values in other regions between the upper and lower limits of the roots. 
In the method of principal axes, a very useful first approximation for each 


x H . H B P d 
root, Bm, of the characteristic equation is — N a?,, for each column m of F, 
т 

21 

when со, Cı, .. . , c; are all positive. 

7 ОБ «| S 5 * i ivisi i 
Table 25 shows the application of the method of synthetic division in 

calculating one of the roots of the equation: 


(53) Bt + 6.9653698? + 10.8104948* + 5.4072038 + .810052 = 0. 


The upper limit of the roots of this equation is any positive number; the 


Жу лакай т : A ~ 

lower limit is — 7.965369. The first. trial value 6’=—5,011317=— W^ аў 
: B H B = 

as found in a prineipal-axis problem. Since f(8’) is negative, 8” = - 5.020000 


was chosen arbitrarily to secure a 


s positive value of f(8"). Linear interpo- 
lation between f(8’) E ; 


and f(8") gave the third trial value, 8" = —5.019710. 


* L. E. Dickson, First Course in the Th Di ions (New York: Wi 
Sons, 1922), pp. 21-23. : n the Theory of Equations (New York: John Wiley & 


T Ibid., p. 12. 1 Ibid., pp. 13-15. § Ibid., p. 67 
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Linear interpolation between f(8"^) and f(8") gave the fourth trial value 

B" = —5.019712, for which the value of the polynomial is +-.000018. 
Hence, one root of this equation is —5.019712. 

The values of f(8) in each row of Table 25 were determined by the equa- 

; tions in their corresponding rows of Table 24. In actual application, it is 

| possible to carry out the г calculations in each row on a calculating machine 

without recording any of the values except the rth one in the column 


headed /(8). 


Table 24 


pe a e 8 AB 
Trial = == Trial 8 
со e e: c3 сє 
1 cate: d c - cig' — ci pis 
2 з-с. cy cid cip" = cy P 
3 Lep” = E Du" ley ов" oct Z 
је eat eos!” este! ct" les deci gn metn | gn 
Table 25 
Bt P B | В AB) 
Trial Trial 8 
1.0 6.965369 10.810494 | 5.407203 . 840052 
1 1.018120 — 058806 | —5.011317 
2 1.044742 .023810 | —5.020000 
3 1.043850 — 000142 | —5.019710 
.000018 | —5.019712 


1.043856 


A method of computing the square root on a calculating machine 
Newton's* iterative method of determining square roots may be used 
very advantageously on the calculating machine, using Barlow's Tables to 
determine the first trial value of the square root. 
Let N be the number whose square root is desired, and let x; be 
е v N derived from Barlow's Tables. Determine AN /zo on 


the first 


trial value for th 
the ealeulating machine, and record it. 
Compute a new trial value, х= .5(х-Е М/х) by determining the cumula- 


tive sum of the two products, (.5a%) and [.5(N /z9)]. Determine N/a. If 
the two values z; and N/a are the same to as many decimals as desired, 
then VN 2z; 2 N/zi. If these two values differ, this process may be re- 
peated for as many trials as are necessary to find that trial value x which 
agrees with N/a to the required number of decimals. 


* E. T. Whittaker and G. Robinson, The Calculus of Observations (London: Blackie & 


Son, 1926), pp. 79-80. 
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EXAMPLE 

N = 42.072. 

r9 = | 42.07 in Barlow's Tables = 6.4861391 . 
N = 0.48644738 . 
To 

z JN 5 4862932. 
Tti = .5% + 5( ) = 6.48629324 . 
то 

a = 6.48629324 . 
Tı 


The computation of an inverse 


The outline that was described in a previous section for computing an 
inverse is adequate for small matrices of order 2X2 or 3X3, but it is not 
economical for larger matrices. The outline of computations which will be 
described in this section is applicable to matrices of any order, and the 
computational labor is not prohibitive even when the order is as large as 
10 or 12.* . . 

The method is designed to find the inverse of a symmetric matrix, which 
may be denoted M. If it is desired to find the inverse of a non-symmietric 
matrix P, one can proceed as follows: Compute the matrix РР = M, which 
is symmetric. Find the inverse M- of the symmetrie matrix M by the 


method to be deseribed here. ‘Then compute the matrix product. (MPY. 
We then have 


(54) (MAP) = ((PP')9Apy 
= [Pyapap)yr 
= [any 
= Pä, 

Hence the method can be adapted to find the 


either symmetric or non-symmetric. 


In Table 26 we have a numerical example, a symmetric matrix J of 


order 3X3. It is placed in the upper left corner of the table in the first 


inverse of a matrix which is 


* The method of computing an inverse which is described in this section was designed 
by Ledyard Tucker. See also his paper, “A Method for Finding the Inverse of a Matrix,” 
Psychometrika, III, No. 3 (September, 1938), 189-97. 
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three rows of Section A. The steps in computing the inverse are then as 
follows, including cheek columns: 


Table 26 


Section A Section В Section C 
1 2 3 I Jp Шш I п ш 
1 50 IS 36 80 0 0 1.00 0 0 
2 18 80 36 48 .51 .01 60 1.00 02 
3 36 36 56 36 14 .66 45 27 1.00 
Ch 1.64 .66 .67 Ch 2:05 LAr 1.09 
T kés 1.6% 1.58 X, 1.64 .65 .67 3 2.08 1:97 1.02 
1.00 0 0 1.00 — .60 — 29 1.25 —1.18 — .44 
0 1.00 0 0 1.00 — .27 0 1.96 — .41 
0 0 1.00 0 0 1.00 0 O 3.52 
Ch 1:90 .40 44 Ch 125 .78 .67 
= 1,00 1.00 1.00 Уз 1.00 .40 14 X. 1.25 .78 .67 
Ch 2.64 1.00 1.11 Ch 3.30 2.06 1.69 
E юй s 2.64 1.05 1. = 3.80 2.05 1.09 
1.25 1.96 1.52 
(bam) (can) E x 
ii im 1.25 0 0 2.00 —1.06 — .44 
us ip um до Aag 1.96 0 -1.00 2:07 — 41 
er MEC. zw cui 1.58 = ad АТ 5 
1. Record the column sums, Zi, аз shown. А К 
2. In the next three rows of A write an identity matrix. 
3. Record the column sums, Хз, for n A TOWS. 
es У 
4. "Or otal column sums, 2 = Zi 7 ; 
i Record the t the first column of Section A. Here a 


2 P a эру 
5. Inc n I of Section B co fi 
hdi tween the anticipated column sum, Ch, and the 


differentiation is made be 
Actual a 5 А М 

соз eciprocal, 1/01.=1/.80 = 1.25. The entry, bu, is the first di- 
"o b: ind i 1e п К В. ‘Tt is the entry in the first column and in the 
nal entry in Sec the bottom row of B as shown. 


first y cord this reciprocal in 

St row . Recor ACNE oe кы ; 

7 С a ? the values in the first column of Section С. These entries are 
<. Compute the Vé 5 


ы 
(55) ou bal py, à 
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8. Compute the second column of 5 by the formula 
(56) bj; = ајә — (bica), 


where cs; = .60 is constant for the entire column. 
9. Find the reciprocal 1 bee =1/.51 = 1.96. 
10. Compute the second column of € by the formula 


(57) ee biol 1 ). 


boo 
11. Compute the third column of B by the formula 
(58) ba = 


aj — bira = езе. 


where сз = 15 and сз = .27 are constants for the entire column, 
12. Compute the third column of € by the formula 


(59) Cj = byl А ) , 


bas 
where the reciprocal 1/53 is a constant multiplier for the whole column, 
Continue the process in the same manner so that there will be as 


many columns in B and in С as there are columns in A. The generalized 
formulae are, then, 


i 1 

(60) с» = «(7) 

and 

(61) bj = аж — [baci + biera + °°° + bjorn] 


13. When the columns of B and of C have been computed and checked 
by the column sums, the desired inverse M? is 
(62) M = (ym) (Cum)! , 
where (bym) is the square matrix defined by the columns of B and the rows 
of the identity matrix in A. The square matrix (сап) is defined by the 
columns of € and the same rows which are defined by 
in A. Note that it is the transpose of the matrix, (c 
final equation for M~. The computed inverse 


the identity matrix 
am)’, Which is used for the 
is shown in the table. 
Linear dependence of vectors 

Consider two vectors A and B in Figure 4 


. These two vectors define a 
plane, namely, the plane of the figure. These 


two vectors are said to span 
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the plane of the figure. By this is meant that any vector in that plane can 
be written as a linear combination of the given vectors A and В. Any other 
vector in that plane, such as C, can then be written as the linear com- 


bination 
C=ai+bB, 


where the small letters, а and b, represent sealars and the capital letters 
represent vectors. Any vector C in the plane of the figure is determined 
entirely by the numerical values of a and b. Further, all linear combina- 
tions of А and В are confined to the plane of these two vectors. No vector 
outside of that plane ean be described as a linear combination of A and B 


“у Library 


Figure 4 


This reasoning extends to any number of dimensions. Three vectors span 
а three-dimensional space if they are linearly independent. The three vec- 
tors A, B, and C, in Figure 4 are not linearly independent because any one 
of them ean be described as а linear combination of the other two. If three 
they are linearly independent. Then they span 
а three-dimensional space. Any vector in those three dimensions can be 
deseribed as a linear combination of the three independent vectors, 

The interpretation of linear dependence of vectors will bo et rated bya 
numerical example. Let it be required that the vector € shall satisfy three 
conditions, namely, (1) that it shall lie in the plane of the giv en vectors A 
and B, (2) that it shall be orthogonal to B, and (3) that it shall be of unit 


length. | 
The first condition 15 
A and B. Then 


vectors are not coplanar, 


satisfied if we define C as a linear combination of 


Tm -bB, 
(03) @ екы ей 
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where а and Ь are two parameters which are to be determined by the other 
two conditions. The vector C will be orthogonal to B if the scalar product 


(64) CBS 0; 
and C will be of unit length if 
(65) C=], 


Substituting (63) in (64), we have the equation 


(66) (al + bB)B = 0 
or 
(67) a( AB) + bB? = 0 


Substituting (63) in (65), we have 


ll 


(68) аА? + 2ab(A B) + B 


Mquations (07) and (68) can be solved explicitly for a and b, but, in com- 
puting, it is usually simpler to write, first, a vector C^ which is orthogonal 
to B by satisfying (67) without restriction as to its length. Let a = l. 
Then b = —(АВ)/В°. When such a vector C^ has been determined, it 
can be normalized, and it will then satisfy both conditions. 

As a numerical example, consider Table 27 of direction numbers for 


А and B in three dimensions. Then А? = 0.69; B* 20.83; and the scalar prod- 


І. 


Table 27 


р п HI 
"7 р Ё A? = .09 
5 2 dem 
A :3146 (1H) = .61 
Bh es w(N — 
Ы —.6400 (с) = .2417 
У Кесір. = 2.0341 


uct (4B) = 0.61. If weset а= 1.00 


‚ then b= — (A B)/B?= — 0.7349, and then 
the direction numbers of C’ 


can be written by (63) 
row of the table. The cross produe 


these two vectors are orthogonal. Т 
in row C, and then both conditions 
If the direction numbers of € were 
vector Cy, as shown in Figure 4. 
imposed. They differ in th 
С.А is negative. 


‚ as shown in the third 
t of rows B and C’ vanish, and hence 
he row C” is next normalized, as shown 
are satisfied, namely, CB —0 and C?— 1. 
reversed in sign, they would define the 
This vector also sat 


isfies the two conditions 
at the scalar 


product CA is positive, whereas 


CHAPTER I 
THE FACTOR PROBLEM 


On the nature of science 

This volume is concerned with methods of discovering and identifying 
significant categories in psychology and in other sciences. It is therefore of 
interest to consider some phases of science in general that bear on the prob- 
lem of finding a methodology for a psychological science. 

It is the faith of all science that an unlimited number of phenomena can 
be comprehended in terms of a limited number of concepts or ideal con- 
structs. Without this faith no science could ever have any motivation. To 
deny this faith is to affirm the primary chaos of nature and the consequent 
futility of scientific effort. The constructs.in terms of which natural phe- 
nomena аге comprehended/are man-made inventions. To discover a scien- 
tifie law is merely to discover that a man-made scheme serves to unify, and 
thereby to simplify, comprehension of a certain class of natural phenomena. 
A scientific law is not to be thought of as having an independent existence 
whieh some scientist is fortunate to stumble upon. A scientific law is not 
a part of nature. It is only a way of comprehending nature. 

A simple example is the concept “forces” No one has ever seen a force. 
Only the movement of objects is seen. The faith of science is that some 
schematic representation is possible by which complexities of movement 
can be conceptually unified into an order. The error of a literal interpreta- 
tion of a foree vector as the pictorial representation of a corresponding 
physical entity is seen in the resolution of forces. If a particle moves with 
uniform aceeleration in a certain direction, it is, of course, possible to de- 
scribe the movement by one force, or by two, or by three or more coplanar 
forces, This resolution of a movement into several simultaneous and super- 
imposed movements is frequently done in order that aconvenient and habit- 
ual reference frame may be retained. While the ideal constructs of science 
do not imply physical r oality, they do not deny the possibility of some de- 

É with physical reality. But this is a philosophical 
side the domain of science. 

Consider, as another example, Coulomb’s inverse-square law of electrical 
attraction, A postulated force is expressed as а Eon of the linear sepa- 
ration of the charges. Now, if the charges were to be d they would 
probably be mueh surprised that their actions were berig е «ешр in terms 
of their linear separations. No one assumes that трен а gig Бн the 
charges, but Coulomb's law implies that tie he sra i : ps is to be 
used in our simplified scheme of comprehending the postulated charges, It 


51 


gree of correspondence 
problem that is quite out 
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is more likely that the whole space surrounding the charges is involved in 
the phenomena of attraction and that Coulomb’s law is a fortunate short 
cut for representing approximately a part of the phenomena that are called 
“charges” and “attractions.” It is not unlikely that all these entities will 
eventually vanish as such and become only aspects of an order more in- 
volved than Coulomb’s law implies but not so chaotic as to individualize 
completely every moment of nature. 

A science of psychology will deal with the activities of people as its cen- 
tral theme. A large class of human activity is that which differentiates in- 
dividuals as regards their overt accomplishments. Just as it is convenient to 
postulate physical forces in describing the movements of physical objects, 
so it is also natural to postulate abilities and their absence as primary 
‘auses of the successful completion of a task by some individuals and of the 
failure of other individuals in the same task. 

The criterion by which a new ideal construct in science is accepted or 
rejected is the degree to which it facilitates the comprehension of a class of 
phenomena which can be thought of as examples of a single construct rather 
than as individualized events. It is in this sense that the chief object of 
science is to minimize mental effort. But in order that this reduction may 
be accepted as science, it must be demonstrated, either explicitly or by im- 
plication, that the number of degrees of freedom of the construct is smaller 
than the number of degrees of freedom of the phenomena that the reduc- 
tion is expected to subsume. Consider, as an example, any situation in 
which a rational equation is proposed as the law governing the relation be- 
tween two variables. If three observations have been made and if the 
proposed equation has three independent parameters, then the number of 
degrees of freedom of the phenomena is the 
of freedom of the equation, and hence the formulation remains undemon- 
strated. If, on the other hand, one hundred experimentally independent 
observations are subsumed by a rational equation with three parameters, 
then the demonstration can be of scientific interest. The convincingness of 
a hypothesis can be gauged inversely by the ratio of its number of degrees 
of freedom to that of the phenomena which it has demonstrably covered. 
It is in the nature of science that no scientific law сап ever be proved to be 
night. It can only be shown to be plausible. The laws of science are not 
immutable. They are only human efforts tow 
prehension of nature. 

If abilities are to be postulated as primary 
In overt accomplishment, then the widely ‹ 
viduals must be demonstrable 
abilities. This implies th 
number of faculties. 


same as the number of degrees 


ard parsimony in the com- 


causes of individual differences 
lifferent achievements of indi- 
functions of a limited number of reference 
at individuals will be described in terms of a limited 
This is contrary to the erroneous contention that, since 
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every person is different from every other person in the world, people must 
not. be classified and labeled. 

Each generalization in the scientific description of nature results in a loss 
in the extent to which the ideal constructs of science match the individual 
events of experience. This is illustrated by simple experiments with a pen- 
dulum, in which the mass, the period, and the locus of the center of gravity 
with reference to 2 fulerum are involved in the ideal construct that leads to 
experimental verification. But the construct matches only incompletely the 
corresponding experimental situation. The construct says nothing about 
the rusty setserew and other extrancous detail. From the viewpoint of im- 
mediate experience, scientific description is nece ssarily incomplete. The sci- 
entist always finds his constructs immersed in the irrelevancies of experi- 


ence. It seems appropriate to acknowledge this characteristic of science, in 
view of the fact that it is a rather common notion that the scientific descrip- 
tion of a person is not valid unless the so-called “total situation" has been 
engulfed. A study of people does not become scientifie because it attempts 
to be complete, nor is it invalid because it is restricted. The scientifie de- 


scription of a person will be as incomplete from the viewpoint of common 
s in scientifie context. 


sense as the description of other object 

The development of scientific analysis in a new class of phenomena usual- 
ly meets with resistance. The faith of science that nature can be compre- 
hended in terms of an order acknowledges no limitation whatever as re- 
gards classes of phenomena. But scientists are not free from prejudice 
against the extension of their faith to realms not habitually comprehended 
in the scientific order. Examples of this resistance are numerous. It is not 
infrequent for a competent physical scientist to declare his belief that the 
phenomena of living objects are, at least in some subtle way, beyond the 


reach of rigorous scientific or 
One of the forms in whicl 


'der. 

| this resistance appears is the assertion that, 
since a scientifie construct does not cover all enumerable details of a class of 
phenomena, it is therefore to be judged inapplicable. Since the analysis of 
cell growth by mathematical and physical principles does not cover every- 
thing that is known about cells, the biologist judges the analysis to be in- 
applicable. Since no mathematical analysis that can be conceived would 
cover all the subtle mysteries of personality, this realm is frequently judged 
to be outside the domain of rigorous science. But physical scientists accept 
ntifie analyses about physical events that leave fully as much 
onstructs. Every explosion in the world has been 
ysion, and no physicist can write equations 
to cover all the detail of any explosive event. It is certain that no two thun- 
derstorms have been exactly alike, and yet fno constructs of physics are 
applied in comprehending thunder and lightning, Novus any demand that 
the detail of the landseape be covered by the same scientific constructs, 


rigorous scie 
beyond the scientific ¢ 
different. from every other expl 
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The attitudes of people on a controversial social issue have been appraised 
by allocating each person to a point on a linear continuum as regards his 
favorable or his unfavorable affect toward the psychological object. Some 
social scientists have objected because two individuals may have the same 
attitude score toward, say, pacifism, and yet be totally different in their 
backgrounds and in the causes of their similar social views. If such critics 
were consistent, they would object also to the statement that two men have 
identical incomes, for one of them earns while the other one steals. They 
should also object to the statement that two men 
The comparison should be held invalid because one of the men is fat and 
the other is thin. This is again the resistance against invading with the 
generalizing and simplifying constructs of science a realm which is habitual- 
ly comprehended only in terms of innumerable and individualized detail. 
Every scientific construct limits itself to specified variables without any 
pretense of covering those aspects of a class of phenomena about which it 
has said nothing. As regards this characteristic of science, there is no differ- 
ence between the scientific st udy of physical events and the scientific study 
of biological and psychological events. What is not generally understood, 
even by many scientists, is that no scientific law is ever intended to repre- 
sent any event pictorially. The law is only an abstraction from the experi- 
mental situation. No experiment is ever completely repeated. 

There is an unlimited number of ways in which nature сап be compre- 
hended in terms of fundamental scientific concepts. One of the simplest 
ways in which a class of phenomena can be comprehended in terms of a 
limited number of concepts is probably that in which a linear attribute of an 


event is expressed as a linear function of primary causes. Even when the 


relations are preferably non-linear and mathematically involved, it is fre- 
quently possible to use the simple linear forms as first approximations. A 
well-known example of this type of relation is that in which the chroma of 
a spectral color is expressed as a linear function of two arbitrarily chosen 
primaries. If two spectral colors are chosen arbitrarily for use as primaries, 
it is possible to express any intermediate color as a linear function of the 
two arbitrarily chosen primaries. The coefficients of the two terms of this 
linear function represent the angular sizes of the two sectors into which a 
color rotator is divided. When the rotator is spun, the intermediate color 
15 seen. But here, as elsewhere in science, although the chroma of the re- 
sulting color is expressed in terms of the linear function of the arbitrary 
primaries, it does not follow that the saturation and the gray-values are 
expressed by the same law. There is still debate about which colors are to 
be considered primary. This question can be settled only by discovering 
that a certain set of primaries gives the most, parsimonious comprehension 
of some phase of color vision. A parallel in the delineation of human traits 
is their description, in first approximation, as linear functions of a limited 


are of the same height. 
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number of reference traits. The final choice of a set of primary reference 
traits or faculties must be made in terms of the discovery that a particular 
set of reference traits renders most parsimonious our comprehension of a 
great variety of human traits. 


The purpose of factor analysis 

A factor problem starts with the hope or conviction that a certain do- 
main is not so chaotic as it looks. The range of phenomena that is repre- 
sented in any factor analysis will be referred to as its domain. Ifa particular 
investigation is limited to measurements in visual perception, it is likely 
that auditory effects will be outside of its domain. The factorial methods 
were developed primarily for the purpose of identifying the principal di- 
mensions or categories of mentality; but the methods are general, so that 


they have been fond useful for other psychological problems and in other 
: can be regarded as a general scientific 


sciences as well. Factor analy 
method. Since the methods were developed especially for the solution of 
psychological problems and since the new methods have been used so far 
mainly on psychological problems, these will be used for most of the ex- 
amples in this text. Some of the principles can be illustrated to best advan- 
simple mechanical or geometrical examples; and these will 
ecially when it is desired to illustrate a logical prin- 
istractions of controversial or nebulous sub- 


tage in terms of 
be used occasionally, esp 
ciple without involving the d 
ject matter. 

The factorial me 
ences among people, 


thods were developed for the study of individual differ- 
but the individual differences may be regarded as an 
avenue of approach to the study of the processes which underlie these dif- 
ferences. If a process is invariant in all its characteris es in an experimental 
population of individuals, then there exist no individual differences as re- 
gards such a process, and it cannot be investigated by factorial means. 
Thus, if we select an experimental population of individuals who are all 
equally good or equally bad in some form of herd e on then we can- 
not expect to identify or differentiate such PRESES y actorial methods. 
It is only to the extent that the individuals of an experimental population 
exhibit individual differences ina process and its effects that these effects 
can become accessible to investigation by factorial methods. mE 
When a particular domain is to be шеш һу ee of individual 
differences, one can proceed in one of two — р n е ent " hypoth- 
esis regarding the processes that underlie the ind ^ kw : А » suem, алй опе 
can then set пра factorial experiment, or a vine 0 і n = atory experi- 
ment, to test the hypothesis. If no PEATE y p» а ae ailable, one 
can represent the domain as adequately as den Р ie 2а of a set of 
Measurements or numerical indices and em s ae e d p experiment. 
The analysis might reveal an underlying order which would be of great as- 
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sistance in formulating the scientific concepts covering the particular do- 
main. In the first case we start with a hypothesis that determines the n: 
ture of the measurements that enter into the factorial analysis. In the sec- 
ond case we start with no hypothesis, but we proceed, instead, with a set of 
measurements or indices that cover the domain, hoping to discover in the 
factorial analysis the nature of the underlying order. It is this latter appli- 
cation of the factorial methods that is sometimes referred to as an attempt 
to lift ourselves by our own boot straps, bec: 


ause the underlying order in a 
domain can be discovered without first postulating it in the form of a 
hypothes 


в. This is probably the characteristic of factor analys 
it some interest as a general scientific method. 
"actor analysis 


s that gives 


8 not restricted by assumptions regarding the nature of 
the factors, whether they be physiological or social 
correlated or uncorrelated, For example, some 
to be defined by endocrinological effects. Others may be defined in bio- 
chemical or biophysical parameters of the body fluids or of the central nerv- 
ous system. Other factors may be defined by neurological or vascular rela- 
tions in some anatomical locus; still other factors may involve parameters 


in the dynamics of the autonomic nervous System; still others may be de- 
fined in terms of experience and schooling, F 
variety of phenomena within a domain 


termined, at least in part, by a relatively small number of functional unities 
or factors. The factors may be called by different names, such as “causes,” 
“faculties,” “parameters,” “functional unities,” “abilities,” or “independ- 
ent measurements." The name for a factor depends on the context, on one’s 
Philosophical preferences and manner of speech, and on how much one al- 
ready knows about the domain to be investigated. The factors in psycho- 
logical investigations are not ordinarily to be thought of as elemental things 
which are present or absent, like heads or tails in the 
The exploratory nature of factor analysis is ofte 
tor analysis has its principal usefulness 
naturally superseded by rational formulations in terms of the science in- 
volved. Factor analysis is useful, especially in those domains where basic 
and fruitful concepts are essentially lacking and where crucial experiments 
have been difficult to conceive. The new methods have a humble role. They 
sas us to make only the crudest first map of a new domain. But if we 
Nine de! in wil enable ue o the rough factorial map 
m s d beyond the exploratory factorial 


stage to the more direct forms of psychological experimentation in the 
laboratory, 


, elemental or complex, 
of the factors may turn out 


actor analysis assumes that a 
are related and that they are de- 


tossing of coins. 
n not understood. Fac- 
at the border line of science. It is 


In a domain where 
formulated and tested 
cept for didactic 


fundamental and fruitful concepts are 


; it would be absurd to use the factorial 
purposes to illustrate factorial logic. 


already well 
methods ex- 
In such situations 
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there are available more direct methods of investigating rival hypotheses. 
In the relatively young sciences and in the new domains of the older scien- 
ces, the factorial experiments will be useful. It seems quite likely that the 
new methods will be applied with profit in the field of meteorology, but it 
is not likely that they will ever be used in classical mechanics. 

In factorial investigations of mentality we proceed on the assumption 
that mind is structured somehow, that mind is not a patternless mosaic of 
an infinite number of elements without functional groupings. The extreme, 
opposite view would be to hold that mind has no structure at all. In the 
interpretation of mind we assume that mental phenomena can be identified in 
terms of distinguishable functions, which do not all participate equally in every- 
thing that mind does. It is these functional unities that we are looking for 
with the aid of factorial methods. It is our scientific faith that such dis- 
tinguishable mental functions can be identified and that they will be veri- 
fied in different types of experimental study. No assumption is made about 


the nature of these functions, whether they are native or acquired or wheth- 


er they have a cortical locus. 

In order to illustrate the met hod, let us consider a set of gymnastie stunts 
that might be given to a group of several hundred boys of comparable age. 
A factor analysis starts with a table of intercorrelations of the variables. 
different stunts, we should have a square 20X20 table 
of every performance with every other performance. 
hether these relations can be compre- 


If there were twenty 
showing the correlation 
Our question now is to determine W ese rela 

ed in terms of some underlying order, which is simpler than the whole 


al hundred experimentally determined coefficients of correla- 
that some of the stunts require principally strength of 
5 require principally a good sense of balance, that 
1 of bodily movement. Several tests that require 
good sense of balance might not require arm strength, while those which 
require a strong arm might require very little bodily balance. We might then 
find that the correlations ean be comprehended in terms of a small number 
of functional unities, such as sense of balance, arm strength, or speed of 
bodily movement. Each of the gymnastic tests might require one or several 
of these functional unities; but it is not likely that every test will require 
every one of the functional unities that are represented by the whole set of 
gymnastic tests. A factorial analysis would reveal these functional unities, 
anid weavouldl say that eack of tems. розу factor in the battery of 

R f these functional unities, such as 


tests. Now, if we should take any one o ' а | 
sense of b. dan e, md represent it in a new set of twenty tests of great variety 
sense glance, а ¥ 


which all required bodily balance, we might ant that е several 
primary factors involved in this domain. I m E 2 Е might con- 
ceivably be a separate balancing factor се А ae men тапа canals, 
or there might be some other breakdown of the balancing factors that would 


hend 
table of sever: 
tion. Let us suppose 
the right arm, that other 
still others require speec 
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be revealed in an extensive study of balancing test A new set of more 
refined primary factors might be found within the domain of bodily balance. 
This process might continue with the factorial investigation of more and 
more restricted domains, as long as the functional unities continued to be 
difficult to conceive in direct experimentation. Eventually, the factorial 
methods, which are essentially exploratory, would yield to the reformula- 
tion of a problem in terms of the fundamental rational constructs of the 
science involved. It is not unlikely that factorial analyses will point the 
way in the work of inventing significant and fundamental scientific concepts. 

Let us consider, next, an example in the sensory and perceptual fields. 
Let us start with a set of twenty perceptual tests involving several of the 
modalities. Some of the tests might require visual acuity; others would re- 
quire keen discrimination of rhythm; still others might require speed of per- 
ception. Each of the perceptual tests might involve one or more of these 
functional unities; but few would require all these functions. Some of the 
tests, for example, might not depend on visual acuity. In this simple case 


we should not be surprised to find factorially the primary functional unities 
that are obvious at the start. 


If we turn to the more centr 
tual and temperamental 
Suppose that here also w 
some day be as obvious 
to us now. 


Our work in the factor 


al functions that are involved in the intellec- 
differences among people, it seems reasonable to 
¢ may expect to find functional unities that will 
as the sensory and perceptual unities are obvious 


: ial study of the human mind rests on the assump- 
tion that mind represents a dynamical system which can eventually be 
understood in terms of a finite number of parameters. We have assumed, 
further, that all these parameters, or groups of parameters, are not in- 
volved in the individual differences of every kind of mental teeta Just as 
we take it for granted that the individual differences in visual acuity are 
not involved in pitch discrimination, so we assume that in intellectual tasks 
some mental or cortical functions are not involved in every task. This is 


the principle of “si structure” op tie x У Я 
А Į ple o simple structure” or “simple configuration” in the under- 
lying order for any given set of attributes, 


Observation and educational experience 


tion that the mental abilities are determined by a great multiplicity of 
causes or determiners and that these determiners are more or less structured 
or linked in groups. This multiplicity of determiners can þe thought of as 
a field of elements in which all are not equally closely linked. Some = жиы 
тау һе quite independent in their actions, while others may bu rather closely 
associated. The factors are probably functional groupings, and it would 


a 1 TM 1 

be a distortion to assume that they must be elemental. We know precious 
* Several such factors mi 

common-factor variance in 


lend plausibility to the concep- 


ght appear in the place of speci "i i 
z ifie variance and uninter| ore ed 
the earlier studies, Xa 
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little about the determiners of human talent and temperament, and we 
should not impose upon our thinking an unnecessarily rigid causal frame. 

If we grant that men are not all equal in intellectual endowment and in 
temperament and if we have the faith that this domain can be investigated 
as science, then we must make the plausible and inevitable assumption that 
individual differences among men can be conceived in terms of a finite num- 
ber of traits, parameters, or factors. Some of the factors may be found to 
be anatomically determined; others will be physiological; while others will be 
defined, at first, in experiential, educational, and social terms. As scientists, 
we must believe that a set of categories can be found for the understanding 
of mentality, which have, by their simplicity, a prior claim on our concep- 


tual formulations. 


Factor analysis and empirical prediction 

Factor analysis involves a number of well-known statistical procedures, 
and it is only natural for the student to begin his study of factor theory with 
a statistical point of view. In some respects this point of view is legitimate 
and useful, but there are some fundamental differences between the objec- 
and the customary objectives in statistical work 


tives of factor analys 
that might as well be made explicit at the start. Some of the controversial 


questions in factor theory have their source in misunderstandings about the 


objectives for which factor theory was developed. | 

Many statistical problems take a form in which а certain number of in- 
'e available for each member of a statistical population, and it is de- 
sired to predict some new index in terms of the known or given indices for 
each member of the population. A familiar example in psychological work 
is the prediction of student scholarship in terms of indices that are available 
at the time of college entrance. Prediction problems of this type are re- 
Solved by writing a regression equation, im whieh the dependent variable 
which is to be predicted is expressed asa linear function of the independent 
variables. The weight given to each independent variable is called its “те. 
gression coefficient,” and the weights are so determined as to minimize the 
sum of the squares of the residual errors. The multiple correlation coeffi- 
cient is the correlation between the predicted values of the dependent vari- 
able and the actually observed values. When the residual errors are small, 
the multiple correlation is high. Factor analysis ашыш from these statisti- 
cal problems in that there is no distinction between independent and 
dependent variables. In factor analysis one does not tele same one vari- 
able which is to be predicted or determined by the other variables. All the 
variables in factor analysis are tres 


ated alike in this sense. Whenever the 
investigator pivots his attention on one of the given variables which is cen- 
ju P ? d which is to be predicted by a set of independent 


ral in i rtance an Ё : s 
poti. he is not talking about a factor problem. He is then talking 
ünables, 5 р 08 


dices аге g 
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terpretation would agree with the usual meaning of the terms, “independent ad 
and “dependent,” if the factors had previously been isolated and if we had 
started out to determine which tests constitute the best appraisal of a given 
factor. Such a problem would be similar to the familiar problem in test con- 
struction when the object is to determine the relative validities of several 
tests for a given criterion. In the exploratory type of problem for which the 
factorial methods were developed, the object is to discover the underlying 
order in a system of variables and to identify their nature. In this use of the 
factorial methods there is no distinction bet ween independent 


and depend- 
ent variables. 


Psychological postulates and definitions 

The factorial methods have been developed primarily for the purpose of 
analyzing the relations of human traits. These are defined as 

Definition 1. 4 trait is any attribute of an individual. 

The factorial methods are applicable also in the analysis of attributes of 
inanimate members of a group. The me 
may be moments in time or теш 
which has a set of attributes. 
plicitly, but it is implied in the 
been developed primarily with 
be explicitly discussed, even tl 
problems which involve the 
group. 

It is useful to distinguish between those traits which are descriptive of 
the individual as he appears to others and those tr. 
primarily in the things that he c 
definition of “ability.” 

Definition 2, 

can do. 
This definition implies that there are 
able things that individuals can do, 


follows: 


mbers of a statistical population 
ions in space or any other entities, each of 
This generalization will not be made ex- 
following chapters. Since the methods have 
psychological categories in mind, these will 
1ough the same methods are applicable to 
attributes of inanimate members of a statistical 


tits which are exemplified 
an do. This distinction is involved in the 


An ability is a trait which is defined by what an individual 


as many abilities as there are enumer- 
e: ШШ t Each ability is therefore objectively 
defined in terms of a Specified task and of a specified method of appraising it. 


Definition 3. The task, together with the method of appraising it, which 
defines an ability, is called a test. 
Definition 4. The nun 


Score. 

It is implied in these definitions t} 
the score in the test Which defines th 
is covariant with the true se 
It is desirable to develop tl 
are independent of the assur 
experimental population. 7n 


verical evaluation of а test performance is called a 


iat an index of ability is covariant with 


5 the ability and that a true index of ability 
ore in the test. 


ле factori 
mption of 
the presen 


al methods in such a manner that they 
normality of ability in any particular 
t theoretical development of the factorial 
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methods it will not be assumed that any of the distributions of ability are nor- 
mal. 

The application of the factorial methods in science rests on a fundamental 
postulate, 

Postulate. The standard scores of all individuals in an unlimited number of 
abilities can be expressed, in first approximation, as linear functions of 
their standard scores in a limited number of abilities. 

The correlation between the true scores in two tests will be referred to as 
the correlation between the two abilities which are defined by the tests. In 
statistical work it is customary to refer to two variables as “independent” 
when their correlation is zero. The term independence will be used with three 
different meanings. They will be designated by appropriate adjectives un- 
less the context makes the designation unnecessary. 

Definition 5. A set of n abilities are linearly independent if the rank of 

the matrix of their true intercorrelations is n. 
Definition 6. Two abilities are statistically independent in a population 


if their correlation is zero in that population. | 
Definition 7. Two observations are experimentally independent if they 
are experimentally distinct, во that one is not derived from the other by a 
'onstraint either of the experimental situation or of the computations. 
In one sense, no two observations can ever be experimentally independ- 
ent. The term can be used only with reference to the state of knowledge at 
the time that the observations are made. | u 
It is clarifying to interpret geometrically the relations of abilities. In 
Such a context, two abilities that are uncorrelated in a population will be 
called orthogonal in that population. Two abilities that are correlated in a 
population will be called oblique in that population. "uM | 
There is special interest in the limited number of abilities in terms of 
which all other abilities can be defined, since these are landmarks in terms 


of which all abilities can be comprehended. 


The reduction of raw scores to standard scores | | 
ts, ordinarily, with a table of intercorrelations be- 
“у 


are to be analyzed. These correlations are com- 
t scores for the experimental population. It is 
of interest, therefore, to consider the кш е = з to derived 
Scores, such as standard scores, and the effect ree : = it ions on the cor- 
relation coefficients and on the resulting factoria вне ure. 

e coefficient of correlation between two 


i 3 to recall that th 
oo ets “naffected if an arbitrary constant is added to the a’s 


: adde the y’s. Furthermore, the coef- 
: ‘itrary constant is added to y | оге, бе) 
and if some arbitrary ¢ he two variables is unaffected if the 2’s are 


fici relation between t j i چ‎ fie } 
үө eon arbitrary multiplier and if the y’s are similarly subjected 


A factor analysis star 
tween the variables that 
puted from the individual tes 
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to some arbitrary stretching factor. These properties of the correlation coef- 
ficient can be summarized in the statement that the correlation «ашы 
for two variables т and y is the same as the correlation between any linear 
functions of 7 and y. Stated in symbols, we have 

(1) Try = Таз) (еу) » 

where a, b, c, d, are arbitrary constants. ' 

Two ways of reducing the given raw scores to standard form will be con- 
sidered here. They give numerically different values for the standard scores 
and different intercorrelations. en | 

Case 1, assuming that the distribution of ability is not necessarily Gaussian: 
Let the given distribution of raw scores be represented by the skewed fre- 
quency curve at A in Figure 1, with mean at ms, which is the mean of the 
raw scores in test j. Let Ау denote the raw score of individual ¿in test j. 
The frequency distribution A of the raw scores Y j; will be subjected to two 
changes, namely, (1) a translation to the position B without change in 
shape, so that the new mean will be at the origin O, and (2) a stretching fac- 


tor, by which the distribution B is changed to the form C, which has unit 
standard deviation and mean at the 
of the raw scores X jis changed by 
tion of scores (X 
the stretching f. 


origin. The frequency distribution А 
the translation to the frequency distribu- 
ji—mj), as shown at B. This distribution is changed by 
actor to the distribution of standard scores, 


Sa = =" (Nj — mj, 
as shown at C. The translation is ove 


r the distance mj, and the stretching 
factor is 1/o;. The new distribution 


at C has its mean at the origin, and it 
has unit standard deviation. The shape of the distribution C is exactly the 
same as the shape of the distribution at A 
The correlation of any variable with the r 
correlation with the corresponding st 
ture of this reduction to standard se 
unaltered and the correlation coefficients are unaltered, 

Case 2, assuming that the distribution of true вс 
If the investigator wants to make this 
scores А 


as regards degree of skewness. 
aw scores Ху is the same as the 
andard scores Sji The important fea- 
ores is that the degree of skewness is 


ores in ability j is Gaussian: 
assumption, the reduction of the raw 
an additional cha 
al distribution form, 
are not the same as in the first, 
tween the variables are 
simply to regard the raw scores 
cording to the ability 


ji to standard scores involves 
distribution, namely, to the norm 
of the standard scores 
tion coefficients be 


апке in the frequency 
The numerical values 
case, and the correla- 
not the same. The 
as placing the individu 
represented by the test. If the 


procedure is 
als in rank order ac- 
percentile rank of each 


1 HADI 
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individual is determined from his raw score, the corresponding normalized 
standard score can be found directly and without further computation mam 
the Kelley-Wood tables of the probability curve or from similar tables. This 
reduction to normalized standard scores gives a distribution that is normal 
with mean at the origin and with unit standard deviation. The only func- 
tion served by the raw scores in this case is to arrange the individuals in 
rank order, and no attention is then given to the actual numerical values of 
the scores beyond their rank order. | 
These two ways of reducing the raw scores to standard form raise the 
question as to which of the two ways should be adopted. Psychological con- 
siderations enter into this choice, but fortunately the factorial results do not 
seem to be seriously affected in those problems which are otherwise determi- 
nate. In dealing with the raw scores on psychological tests, it must be rec 
ognized that the scoring methods are, on the whole, quite arbitrary. So 
many points are allowed for this, and so many points are deducted for 
something else. The resulting raw scores in tests are as arbitrary as the 
scoring formulae adopted by the intuitions of the investigator, The editorial 
composition of a paper-and-pencil test also has an effect on this problem. If 
a large number of relatively casy tasks is included, the raw scores will rise. 
The dispersion of the scores and the shape of the distribution are markedly 
affected by the more or less arbitrary time limits that are adopted for a 
test. If the distribution is positively skewed, it can ordinarily be made to 
approach the normal shape by merely incr easing the time limit for the tost, 
It takes very little experience in test const ruction to realize that the dis- 
tribution of raw scores has relatively little fundamental significance, and 
consequently it seems proper to regard them as essentially not much more 
than a set of indices that place the subjects in rank order according to their 
ability to do the kind of task that is involved in the test. If the intuitions 


of the psychologist in ass mbling a new test and in adopting a scoring for- 
mula are not too seriously in error, he will obt 


of his subjects that will remain essentially 
in editorial composition and in some ch 
though the raw scores and the shape of their distribution might be subject 
to considerable alteration by such changes. 

If the factorial methods were seriously dependent on the investigator's 
luck in hitting some sort of “right” scoring formula and time limit for each 
test, the new methods would not be able to produce meaningful results. 
The factorial methods are sufficiently powerful that one can take consider- 
able liberties with the raw scores without seriously affecting the results. If 
we take a factor analysis in which several fundamental and meaningful fac- 


tors have been clearly identified, it would be instructive to subject the fac- 
tor methods to a severe test by radical changes in the original raw scores 
and then make a new factor analysis with the altered scores. We might 


ain an approximate rank order 
unaltered with reasonable shifts 
anges of scoring formulae, even 


THE FACTOR PROBLEM 67 


change the scores in one test by writing their reciprocals instead; the scores 
in another test might be altered by writing their square roots; the scores for 
another test might be altered by writing their logarithms; still another test 
might be altered by using the squares of the original scores instead of the 
scores themselves; and so on with a new monotonic function for each test. 
The rank orders of the subjects in a test would be either the same or com- 
pletely reversed. We might use case 1 in reducing the scores to standard 
form so as to retain for the correlations all the distorted distribution shapes. 
The correlation coefficients to be analyzed would then be markedly different 
from those used in the original analysis, but it seems quite likely that the 
same basic factors would be identified. This demonstration has not been 
made,* but it probably would be successful in showing the power of the 
factorial methods in isolating the underlying order among the test variables 
and the basie factors that determine the individual differences. 


* Since this chapter was written, the demonstration has been made, and it is described 


in chap. xv. 


CHAPTER II 
FUNDAMENTAL EQUATIO 


Reference abilities 

In a multiple correlation problem one writes a regression equation to ex- 
press a dependent variable as a linear combination of a set of independent 
variables. Applied to psychological tests, the regression equation is an at- 
tempt to predict a criterion—-the dependent variable—as a linear combina- 
tion of a set of tests—the independent variables. The observation equation 
which is the starting-point for multiple-factor theory is analogous to the 
familiar regression equation in certain respects. The score in a test is ex- 
pressed as a linear function of the scores in a set of postulated reference abil- 
ities. Here the standard score sj; of individual 7 in test j corresponds to the 


N 


©» 
n 
F Р 5 
Factor Matrix Population Matrix Score Matrix 


Ficure 1 


dependent variable which is to be accounted for; but in the factor problem 
all the individual test scores are dependent variables, in this sense, that are 
to be accounted for. 

If an experimental population of N individuals has been given a set of 
n tests, the resulting tabulation of scores will be an nXN-score matrix А, 
whose elements are the standard scores Sji Where the subscript j refers to 
the test and i refers to the individual (see Figure 1). If the test battery has a 
fairly large number of tests, n, and if these cover a fairly restricted range, 
one might attempt to account for the scores sjiin te f 
of factors than the number of tes 
following equation: 


rms of a smaller number 
s. This assumption is represented in the 


(1) Sji = Сахи + сул + салин Ч + ctu, 

where s;; is the standard score of individual 7 in test 7, 

scores of individual i in each of the uncorrel : 
68 


the т?з are standard 
ated reference abilities, and the 
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c's are the weights assigned to the standard scores in the reference abilities 
for the determination of the observed standard score sji. The equation is 
erms in the right-hand member, and it is hoped that 


written to represent q t 
abilities will be relatively small compared with the 


this number of reference 
number of tests, n, in the whole battery. 
The psychological interpretation of equation (1) is that a subject’s per- 
etermined in part by the abilities that are called 
degrees to which the subject possesses 
that the a's in equation (1) contain the 
subscript i but not j. Hence the x's are descriptive of the individuals but not 
of the tests. The c's in the same equation have subseript j but not 7. Hence 
the c's are descriptive of the tests but not of the individual subjects. The 
first term represents the contribution of the first reference ability to the 
score, the second term represents the contribution of the second reference 
ability, and so on, for q reference abilities. If the first reference ability is in- 
volved in the test j, then the coefficient сл will be large and positive. If 
the individual 7 possesses this reference ability to a marked extent, then his 
standard score in that ability, їз will be large and positive. The product 
will be an appreciable contribution toward a good score in the test. If the 
second reference ability is not involved in test j, then the coefficient cj: will 
be zero; and if the individual i has this second reference ability to a marked 
degree, his standard score tei will be large and positive, but it will not con- 
tribute toward the score in the test because the test does not involve this 
second ability. For this individual subject and test, the second term van- 
ishes, Similar interpretation ean be made for high and low test coefficients 
and for high and low standard scores in the reference abilities. 
1f the standard scores of N individuals inn abilities are expressed as linear 
functions of their scores inr lincarly independent abilities, where т «n, then 
the y abilities will be called reference abilities. | | 
It will be shown that if à battery of tests can be described with reference to 
‘lities, there exists an infinite number of sets of r orthogonal 
cw lescription can be made with equal accuracy. 


the ¢ 
onal abilities may be chosen for purposes of de- 


formance, ву, on a test is d 
for by the test and in part by the 
these abilities, It should be noted 


r orthogonal a 
übilities in terms of which 
An arbitrary set of r orthog 

arbitrary set of О! ; pe 
scripti | m ire the statistically independent or orthogonal reference 
f jon. These are t 8 


1 à «evibed in terms of r orthogonal refer- 
abilities ¬ battery of tests can be describec 4 
bilities, If a battery of be described by а set of r oblique reference 


itu reference ability be represented by a 

. While each of the tests used in ex- 
ability, it may happen that the reference abil- 
q individuals are described are not represented 
combinations of several tests. A linear com- 


test in which it is involve 
perimental work defines an 
ities in terms of which tests апо 
by actual tests but by linear € 
bination of tests may be thought 3 

It should be noted that, even if e 


f as a composite test. 
ach individual сап be described in terms 
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of a limited number of independent reference abilities, it is still possible for 
every person to be different from every other person in the world. Each 
person might be described in terms of his standard scores in a limited num- 
ber of independent abilities. The number of permutations of these scores 
would probably be sufficient to guarantee the retention of individualities. 

With a limited number of abilities not only does this formulation allow 
every person to be different. from every other person, but it also allows the 
widest possible differences between several individuals who attain the same 
objective performance in a test. This may be readily seen by considering а 
hypothetical example. Assume that a test calls for two abilities, such as 
ability in abstraction and ability in the manipulation of numbers. Several 
individuals try the test and attain the same score. One of them may possess 
a high degree of ability in making the abstractions involved in the test, but 
he may be slow in numerical manipulation. Another may be slow in formu- 
lating the abstract part of the problem, but he may make up for this de- 
ficiency by superior numerical speed. The objective result might be the 
same. The purpose of factor analysis is to obtain a quantitative description 
of each fundamental ability in cach individual by means of tasks that re- 
quire these abilities in different amounts. Since every task is probably com- 
posite in the abilities required, it is necessary to make the appraisal of the 
abilities of individuals by analytical methods. This is exactly the object of 
the multiple-factor methods as applied to the problem of describing the 
abilities of people. 

Factor analysis is reminiscent of faculty psychology. It is true that the 
object of factor analysis is to discover the mental faculties. But the severe 
restrictions that are imposed by the logic of factor analysis make it an ardu- 
ous task to isolate each new mental faculty, because it is necessary to prove 
that it is called for by the experimental observations. Factor analysis does 
not allow that a new faculty be added as soon as a new name can be found for 
the things that people can do. In order to prove that reasoning and abstrac- 
tion are two different faculties, for example, it will be necessary to show that 
the tasks which call for such activities really do involve two factors, and not 
one. 

In the psychology of the future it may be found useful to postulate a dif- 
ferent form of ideal construct for the description of mental endowment than 
the simple one that is implied in equation (1). The ideal constructs of the 
future may involve elements with location in 
dynamic, and temporal constraints analogous to the ideal constructs of ge- 
netics. It would be unfortunate if some initial success with the analytical 
methods to be described here should lead us to commit ourselves tosthem 
with such force of habit as to disregard the development of entirely difftere 


re 
constructs that may be indicated by improvements in measurement and by 3 


Inconsistencies between theory and experiment. 


a space frame with spatial, 
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The factor matrix and the population matrix 

* Equation (1) ean be represented in rectangular form, as shown in Figure 1. 

The factor matrix F contains the elements су, which are descriptive of 

tests j in terms of reference factors m. The population matrix P contains 
. the elements ты, which are descriptive of individuals i in terms of reference 

factors m. The eross product of a row of F and a column of P is represented 
If the factor matrix P is multiplied by the population ma- 
e score matrix 5 with elements sj; as shown in equa- 
1. The observation equation (1) ean be written in 


in equation (1). 
trix P, the product is th 
tion (1) and in Figure 
the matrix form 


(2) S=FP, 


or in the summational notation 


a 
" 
= Cmt mi ; 


(3) sji 
т=1 

form of Figure t- I will be assumed in our use of 

ference abilities represented by columns of F are 

er the equation will be generalized for the case 


or in the rectangular 
this equation that the re 
Orthogonal. In a later chapt 
lated reference factors. 

starting-point for the development. of 
ed that all the c’s and all the 2’s are 
ıe individual scores sj; in each test at 


of oblique or corre 
This observation equation is th 
factorial theory. It should be not 
Unknown and that we know only tl 
the start of a factorial analysis. 
Interpretation of the test coefficients | " 
of the factor matrix F describe the tests j in terms of the 
alled the test coefficients or factor loadings. Their 
ns of the variance of the tests. 
cores, their sums for the experimental 


The entries cj» 
factors m, and they are e 
an be found in tert 
andard sc 
Then, by (3), 


Interpretation e: 
and Tmi are st 
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Population are identic 


Ср BS 
and 
N 
E NS 0 
(5) ‘ Zoe 
i=l 


= 
Ww 


MULTIPLE-FACTOR ANALYSIS 


The standard deviations of sy; and of ты; are both unity by definition. 
Hence 


ENS 1 
(6) А m a 
^ del 
and 
pus 
= 
(7) Va. th 1 
cen 
The square of the standard Score, Sji, 15 
1 q 4 
° us a Lu ы < 
(8) х = > Суы: > Carta) + N Cimini (т # M), 
m-l М=1 т= 1 


where m and M are subscripts denoting the refe 
m # M. Summing the squares of the stand 
tal population and dividing by №, we have 


rence factors and where 
ard scores over the experimen- 


1 N | N 1 q | N q 
2 Md am “в. 2 DINE 
(9) о? = ae Sji = x 2 Chi У Cty + < NN 
| del ' del mel M=1 Ў 


Rearranging the summations, we have 


OMNES N 4 
Š o ^ 1 м Xx d P 
(10) gj — 2 > Cint у > Жым: + b Gn Zt = 1 
m=1 M =1 ier 5 


i m-1 i=l 
But the correlation between e: 


ach pair of orthogonal reference abilities m 
and M vanishes by definition, so that. 


(11) 


1 ` 
Taa = re Tour; = 0 
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and the a's are standard scores by de 
deviation. Hence, 


finition, so that they have unit standard 
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This equation ean be stated in the theorem that the swm of the squares of the 
test cocflicients of a test for all orthogonal factors is equal to unity. 

It should be recalled that this theorem concerning the test coefficients 
assumes a set of uncorrelated reference abilities. Each test is here represent- 
ed as having unit variance, since the test scores are assumed to be reduced 
to standard form. The square of cach test coefficient, Gm is that part of 
the total variance of test у which is attributable to the reference ability m. 
The sum of these variances is the total unit variance of the test. 

The entries in a factor matrix are sometimes called factor loadings. The 
square of any cell entry in F, such as Gin, 055, Gj, WG, represents the fractional 
part of the total variance of a test attributable to a given factor. We have, 
then, the theorem that each factor loading or test coefficient Cim for orthogonal 
factors is the square root of the variance of lest j attributable to the factor m. 


ue factors 
est coefficient Cjm in Figure 1 can be interpreted as 
ance of test j which is attributable to the refer- 
ence ability or factor m- Psychological knowledge about tests enables us 
to postulate different kinds of factors in F. Some of the factors represent 
abilities that are involved in two or more tests of a battery. These are called 
common factors. Another kind of factor is an ability which is involved in 
only a single test of a battery. These are called specific factors. These two 
types of factors ean be illustrated with a psychological example. In a bat- 
tery of tests there might be only one test whose score depends in part on 
For that battery and for that test the ability to write fast 
Would be a specific factor. Performance in another test might depend in 
part on acuity of hearing, and that ability might not be involved in any 
other test of the battery. It would be a factor specific to one test in the bat- 
or number facility or visualizing, 


tery. Other factors such as word fluency 

y. ar factors, ` i 

might 1 A two or more tests of the battery so that they would be common 
зе in tw р 


factors, 


Common factors and uniq 
The square of each t 
the fractional part of the vari 


Writing speed. 


The common factors d the specific factors vide e of as Bonê fide 
abilities, whose isolation and description pro a a 0 x S шш in But, 
in addition to these cognitive factors, it 15 yel a а е аена deal 
also with fortuitous errors in test scores. = ivre as s E ina 
Particular test for à given population, the addition of fortuitous chance 


É scores WC ild increase h 
errors 1e scores 9 
s to the þes fractior of the observed variance. This error par 
ance would some 11% 1C b "ү 


of the total variance of each test should be ү by an error factor 
in the чя matrix P. Since the fortuitous 1 i" iab E ир, x ee test are, 
in general uncorrelated with the variable Sake sies yis is , e must 
provide in the factor m. кидик осо P5 1e battery. 


atrix one erro 
In Figure 2 we have 


an 


e dispersions so that the true score 


represented six tests with three kinds of factors— 
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the common factors, the specific factors, and the error factors. Sampling 
errors will be discussed in a later section. In Figure 2 we have shown three 
common factors in the first three columns. The letters a, b, and e represent 
non-vanishing cell entries. The blank cells represent zero entries. The com- 
mon factors, I, IT, III, are here shown as if each common factor were pres- 
ent in each test. Later we shall consider the situation in which the common 
factors are not all present in each of the tests. The next block of the factor 
matrix of Figure 2 shows eight specific factors. Two of them occur only in 
the first test. We have labeled them 4 and 5. The third test is also shown 
with two specifics, and each of the other tests is represented with one spe- 
cific factor. Each test is assigned a variable error factor ejjin the third block 
of the factor matrix, 


Complete Factor I. 


FIGURE 2 


The total variance of the first te 


о st in this fictitious example can be ex- 
pressed in the summation 


(14) (ап + afs + ad) + (bis + +е = 1, 


on the assumption that all the factors inv 
into three types—common, specific, 
are shared with any of the other te 
factors is unique for one test. 

The total variance of the test can be div 
part which it shares with other tests in a 
which is unique. We then have 


olved in the test can be divided 
and error. Only the common factors 
sts in the battery. Each of the other 


ided into two parts, namely, that 
particular battery and that part 


(15) bat = hj = communality of test j, 
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where the a’s refer only to common factors and r is the number of common 
factors. The notation A? will be used to denote that part of the total vari- 
ance of a test which is attributable to the common factors. 
Definition: The communality of a test is its common-factor variance. 
The specific factors are those abilities which are involved in only one test 
of a battery. Their variance can be written à 


1 
(16) N bn = specificity of test j , 


where the b’s refer only to specific abilities. 
Definition: The specificity of a test is that part of its total variance which is 
attributable to abilities that are unique for the test in a given battery. 
The specifie factors and the variable error factor are all unique for each 
particular test. The fractional part of the total variance of a test which it 
does not share with any other tests of the battery can be written 


t 
(17) > bja + еў; = uj = uniqueness of test j . 


m=1 


Definition: The uniqueness of a test is that part of its total variance which 
is not shared with any other tests in a battery. 
It will be seen that the uniqueness is the complement of the communality, 
so that 


(48) hj + uj = 1. 


A factor that is specific for a test in one battery may become a common 
factor when the test is moved into another battery. Auditory acuity might 
battery, but it might become a common factor if a test 


be specific in one 
where two or more tests depended on audi- 


were moved into a new battery, 
tory acuity. 

All the unique f. 
tor for each test. 
in Figure 2 would be 


‘actors of a test may be combined into a single unique fac- 
Аз an example, the unique-factor entry for the first test 


М, + bîs Hei = ш, 


as is also shown in equation (17). It will be useful to re-write the factor 
matrix F with two blocks instead of three. These two blocks are the common 
factors and the unique factors. This has been done in Figure 3. 

The pattern of the non-vanishing entries in F should be noted. We have 
shown the common-factor block in F as filled, whereas the unique-factor 
block is a diagonal pattern with zeros in the side entries. This pattern fol- 
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lows from the definition according to which each of the six factors oceurs in 
7 one six tests. 
ree e factor matriz which represents the total unit variance of cach 
test is called a complete factor matrix. y \ 

The complete factorial matrix is denoted F;. The factorial matrices pi 
Figures 2 and 3 are complete factorial matrices. In factor analysis it is ax - 
ful to identify by a special name that part of the factorial mat rix which i ep- 
resents only the common factors. The common-factor block isa matrix ol 
order п Xr for n tests and r common factors. In Figure 3 this matrix is of 
order 6X3 with elements a 


jm. 


Factor Matrix F, 


FIGURE 3 

Definition: A factor matrix which represe 

ance of each test is called a reduced factorial matrix. 

The reduced factorial matrix is denoted F. 

From this definition and equation (1 
of the squares of each row 
h? of test j 

If we postulate a unique factor fore 


total number of linearly independent factors in F, is (n+r), where v is the 
number of common factors. But this formulation of the problem involves 
more factors than tests, a situation that would seem to make the problem 
insoluble. Before a stable solution can be expected by any method of anal- 
ysis, the factor problem must be SO 


formulated that the number of observa- 
tions exceeds the number of independent parameters to be 


nts only the common-factor vari- 


5) we have the theorem that the sum 


j of the reduced factorial matrix F is the comm unality 


ach test, as shown in Figure 3, the 


determined. 
The intercorrelations 

The square table of intercorrelations of n tests is c 
matriz. Such a matrix is illustrated 
Square, and it is symmetric bec 


alled the correlation 
. The correlation matrix is 
Tij. When the correlation matrix 


in Figure 4 
AUSE rj, = 


dg 
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written with unity in the diagonal cells, it is called a complete correlation 
matrix, and it is denoted &;. The correlation matrix can be expressed in 
terms of the matrices that have been defined in previous sections. The cor- 
relation between the standard scores sj; and s+; where the subscripts j and k 


both refer to tests, is 


(19) LESS 


since the standard scores have unit standard deviation. If we write it in the 


form 
1X 
x 
(19а) rin = ү лб, 
"с del 


we imply the multiplication of two score matrices, namely, s; and вд. 
The score matrix with elements sj; is of order n XN and it is denoted S. The 


” 
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Pa 
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FIGURE 4 


score matrix with elements six is the transpose of S and is of order N xn. 


Hence we have ? 
(20) Р; = N SS', 


loq Г, are itv ie 
where the diagonal entries of R, are unity, i.e., 


(21) “лу Л 


By (20) and (2) we have 


d MONI 
(22) Rı = y (FıP)(FP) 
or 

1 "d PP" л, 
(23) Ry = y РРР. 
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The product PP’ in (23) represents the cross products of rows of P. These 
сап be written in the form 


ÁN 
i=] 
or 
(25) РР! = Nfa 


i з or reference abilities m 
where fma is the correlation between the two factors or reference abilities 


^ 
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du s Ay a 
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О 


Factor Matrix Fy 
Ficurr 5 
and M. These correlations 
which are unity. Hence 


(26) 


2 à AN ERE ; 
are zero, except the diagonal self-correlations, 


PPS NF, 
where Z is the identity matrix. 


(27) 


Substituting in (23), we have 
T = FPR}. 


This equation is fundamental in factor an 


alysis. It ean be stated as a funda- 
mental theorem, namely, 


that the product of the complete factorial matrix by 
its transpose is the com plete correlation matrix. 


The complete factorial matrix Fı of Figure 3 c 
of the two matrices shown in Figure 5. The 
where the unique factor loadings h 


an be expressed as a sum 
matrix P, is of order n by (п-т), 
ave been omitted. The matrix Р, іѕ also 
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of order n by (n4-7), where the common factor loadings have been omitted. 
If we superimpose these two matrices, we get a matrix sum of the type shown 
in Figure 3. Hence we write 


(28) Ft Fg = Е. f 


Substituting in (27), we have 


(29) Ry, = (F, + FOL + FQ 
or 
(29a) R= (Р. + PCE + FQ. 


This becomes 
(30) Ry = FF, + Fifa t Ра. + г. Я 


in which F, and Ё„ are both of order n by (n4-7). These four separate prod- 
ucts in (30) are shown in rectangular notation in Figure 6. In order to sim- 
plify Figure 6, it has been particularized to the case of n=4 and r=2. The 
products F,; and РЁ both vanish. Hence we have 


(31) Ri = FFL + FAL. 


The product Г is shown in rectangular notation in Figure б. The prod- 
uct „Їз a square matrix of order n X with diagonal entries u3;. It is that 
part of the correlation matrix which shows the contribution of the unique 
factors to the correlation matrix Ri. 

The product PP; in (31) is shown in rectangular notation in Figure 6. 
This product is that part of the correlational matrix X, which is contributed 
by the common factors. It will be denoted & without subscript. Note that 
R contains the intercorrelations of the tests and that the diagonal elements 
are the communalities instead of unity as in Ri. It is of significance that 
the unique factors contribule only to the diagonal cells in the correlation matrix, 
as illustrated in Figure б. The matrix R will be called the reduced correlation 
matrix because its diagonal elements have been depressed from unity to the 
communalities. 

The product P,P7, where F, is of order n by (n+r), as illustrated in Fig- 
ure 6, is the same as the product FF’, where F is a reduced factor посі e 
of order nXr, as shown in Figure 7. We have, therefore, the important 
equation 
(32) PR =R, 


а. @ ау а, 


а. An сз ax 


FIGURE 6 
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which сап be stated as the fundamental factor theorem: The product of the 
reduced factor matrix by tts transpose їз the reduced correlation matrix. This 
fundamental relation between F and R is shown in rectangular notation in 
Figure 7. 

The complete correlation matrix FR, has now been divided into two parts, 
namely, R and Ru, 


(33) RRTA, 


where Ra is a diagonal matrix. This summation is shown in rectangular no- 
tation in Figure 8. Since each diagonal element u} of №, shows the unique 


FIGURE 7 


FIGURE 8 


variance of test j, the complete correlation matrix R, and the reduced corre- 
lation matrix Ё differ only in the diagonal cells. The diagonals of R contain 
the communalities. 

If we regard the correlation coefficients as the given data and the factor 
matrix F as the desired objective in a factorial analysis, we have n(n -1)/ 
experimentally given coefficients which must exceed the number of linearly 
independent parameters in F. It will be seen that, by limiting ourselves to 
the common factors of F and the corresponding intercorrelations, the factor 
problem becomes determinate, even though we admit the existence of unique 
variance, which is known to exist in all psychological tests. This restriction 
of the factor problem to the common factors with no attempt to account for 
the total variance of each test is one of the turning-points in finding a de- 
terminate solution in factor analysis as a scientific method. 
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In general, the rank of the complete correlation matrix is equal to its 
order, namely, n. Since the reduced correlation matrix R is the product 
FF’, it follows that the rank of 2 cannot exceed the rank of F. The reduced 
factor matrix F is of order n Xr, where r «n. Since the columns of F repre- 
sent linearly independent factors, the rank of F is equal to the number of 
common factors r. Hence we have the important theorem: The number of 
common factors represented in a reduced correlation matrix is equal to the rank 
of the matriz. Experimentally obtained correlation coefficients are subject 
to sampling errors, which constitute a fortuitous element in cach coefficient. 
In the strict sense of the term, the rank of a correlation matrix for experi- 
mental data is always equal to its order, namely, n. The scientific problem 
is to account for as much as possible of the relations between tests by the 
smallest possible number of common factors. This leads necessarily to a 
problem of determining when the residuals are negligible for any given num- 
ber of common factors. The computational problem is made more difficult 
because the communalitics in the diagonal cells of R are unknown. These 
diagonal clements must be estimated at the start of an analy 
the factor matrix F for a given set of correlations in /?. 

А In the mathematical sense, the number of linearly independent, factors 
involved in the correlations between n tests is the rank of the correlation 
matrix, irrespective of what the diagonal clements may be. In general, one 
would limit this reduction of corre : 
correl 


in finding 


Г lations into factors by considering only 
ation matrices whose diagonal elements do not destroy the Gramian 
properties of the correlation matrix. In stating the theorem ‘concerning the 
number of factors in a correlation matrix explicitly for the reduced corre- 
lation matrix, we have in mind the scientific problem of discovering physi- 
cally identifiable factors as distinguished from numerical artifacts which 
might be permissible as mathematical solutions but are not allowable as sci- 
entifically significant solutions in a factor problem. 

The correlation matrix for psychologic 
cells has, in general, a rank that is equal to its order, namely, n. Since we 
have for the complete correlation matrix the relation RE, we should 
expect n linearly independent factors in Pi. But by psychological considera- 
tions we expect (n4-7) —q factors in F, Hence the columns of P, are not 
ma Pa a ae e а af Ry, which is of rank n and order 

BI atrix with as many factors as there are 
tests; but such a numerical solution is not ordinarily of much interest or 
brin. since it represents a condensation of (n+r) factors into n linearly in- 
асаду independent p cholera ганју correspond. to the r 

В 5) gie mmon factors of the intercorrclations. 


This numerical solution with as many factors as there are tests will be de- 
scribed in detail in a later section. 


al tests with unity in the diagonal 
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The reliability coefficient 

The correlation between two administrations of the same test or between 
two tests that are designed to be parallel is called the reliability coeffi- 
cient of the test. An example is the correlation between the scores in two 
spelling tests of one hundred words each, when the two lists have been drawn 
as random samples from a longer vocabulary. If the two tests are truly 
parallel, they must involve the same abilities, common and specific, but the 
variable chance errors in the two tests are independent. This situation is 
represented in Figure 9. 

It is customary in psychological work to write the reliability coefficient 
in the diagonal cells of a correlation matrix. By the present analysis it is 
seen that the diagonal entries of Rı are unity, while the diagonal entries of 
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FIGURE 9 


R are the communalities A}. The relation between the reliability and the 
communality of a test may be shown by considering in detail the factorial 
matrix for a test j, a parallel test 7’, another test X, and its parallel test X’. 
The factorial matrix for these four tests is shown in Figure 9. 

Let there be r common factors in the four tests. Let b? be the specific 
variance in test j. Since j and j' are parallel tests, it is evident that they 
must require the same common abilities and the same specific ability. 
Hence b; is recorded in the same column of Ё in Figure 9for both j and j'. 
For the same reason, b; must be common to tests k and k’, which are paral- 
lel. But the variable errors are uncorrelated by definition, even for parallel 
tests, Hence F, shows a separate error factor for each of the four tests. 

The correlation between the tests j and J’ is the reliability coefficient of 
test j, and it will be denoted rjj. Since у= Г, the reliability of j can be 
found in Figure 9 as the cross product of the first two rows of the matrix. 


We then have 


(34) "jj 
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and the reliability of test k is 


(35) ты = > ant bÈ. 


т=1 


By (34) we have 

(36) rj = hj + bj. 
Since the total variance of j is unity, we have 
(37) rj-l-—ej. 


This equation states the theorem: The reliability of a test is the complement 
of its error variance. 
Since 


(38) hj = 1 — 
and 
(39) 

we have 
(40) 

or 


(41) 


2 à 
hj = r; — b, 


and hence 


(42) hz. 
This inequ 


ality states the theorem: The communality of a test is always small- 
er than the 


reliability except in the limiting case where the specific factor is ab- 
sent, in which case the communality and the reliability are equal. 


It is of interest to note that the factorial methods effect a separation be- 
tween the communality and the uniqueness of each test in a battery but 
that the uniqueness cannot be s 


eparated into its two principal parts— 
specificity and error—by factorial methods. In order to estimate the specific 
variance of a test or its reliability coefficient, a separate experiment must be 
made with a repetition of 


the test or with parallel forms of it. If the reli- 
ability coefficient is determined by independent experiment, the specifi? 
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variance can be estimated as the difference between the reliability and the 
communality. 

An object of psychological inquiry is to isolate an increasing number of 
abilities until the specifie variance of each important test is reduced to a 
minimum. It is not likely that any single test will be completely described 
in terms of the factors which it has in common with those of one battery. 
In order to isolate all the abilities that are called for by a test, it will proba- 
bly be necessary to insert it in several test batteries in succession. The spe- 
cifie variance of a test should be regarded as a challenge; it is that part of 
the total variance of a test which is unique in a particular battery, and hence 
its factorial composition is unknown. In order to test a hypothesis concern- 
ing the abilities which are involved in the specifie variance of a test, the test 
should be combined with others which involve the hypothetical abilities. 
If the specific variance is reduced, the hypothesis is sustained. 

For the next few years it will probably be more interesting to isolate new 
abilities than to reduce the specificity in particular tests. Increased knowl- 
edge of the primary mental abilities will facilitate the type of experiment 
by which the specificities of particular tests may be reduced. It will prob- 
ably be found that a considerable fraction of the total variance of each test 
is attributable to factors of such limited social significance that the com- 
plete elimination of the specificity of each test will not be essential in the 
early stages of the scientific study of human abilities. 


Summary of terminology and notation 
The terminology for the different parts of the variance of a test is sum- 
marized as follows: 
Total variance = 1 = hj + b} + ej = hj + uj; 


Reliability =1-¢; 
Communality -1-uj; 
Specificity : 

Uniqueness te; 

Error variance = ej = еф=1—ту. 


The notation for scores, matrices, and matrix elements is as follows: 


N =number of individuals in experimental population ; 
jk =two subscripts for tests; 
=standard score of individual û in test j; 


SH 

i =subscript for individual in experimental population; 

m, M =two subscripts for arbitrary orthogonal reference abilities or 
factors; 

mi  —standard score of individual 7 in reference ability m; 

Cj. est coefficient of test j for the factor m; 


T — number of common factors in correlation matrix; 
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n =number of tests in the battery; 
rj.  —correlation between tests j and k; 
r,;  -reliability of test j; 
ljm  -—test coefficient of test j in the common factor m; 
^, = сотріеѓе factor matrix for total variance of cach test; 
F =reduced factor matrix for common-factor variance in each test; 
P = population matrix with elements ты; 
b; =specific factor loading in test ES 
ej =error factor loading in test j; 
u; — —unique factor loading in test E 
S =score matrix with elements s;;; 
9;  =total variance of test j=1; 
cA = variance of scores Tmi= 1 Б 
R, 


=complete correlation matrix with unity in diagonal cells; 
=reduced correlation matrix with communalities in diagonal cells. 


CHAPTER III 
GEOMETRICAL MODELS 


Geometrical representation in factor analysis 

In understanding the relation between the factor matrix F and the corre- 
lation matrix X, it will be helpful to consider a set of geometrical models. 
The algebraic development of factor theory could be represented by several 
different geometrical analogies, but we have selected one that seems to be 
the simplest and most direct, and it will be used throughout this text. In 
this chapter we shall consider the geometrical representation of problems 
that involve only two common factors. These problems can all be represent- 
ed by two-dimensional diagrams. In later chapters we shall extend the same 
kind of geometrical interpretation to problems involving three common 
factors which can be represented by three-dimensional models. The same 
geometrical interpretations will then be extended to the n-dimensional 
methods of multiple-factor analysis. Most students find it very helpful to 
visualize the problems of factor analysis by geometrical representations. 


Factorial structure 

In Figure 1 we have represented a problem that involves eight tests with 
two common factors. The correlational matrix R is shown at Ze in the figure 
and the factor matrix F is shown at 1b. 

The factor matrix F will now be given a simple geometrical interpreta- 
tion. Let the entries in each row represent the co-ordinates of a point in the 
diagram at Та in the figure. The first row gives the factor loadings .70 and 
.30 for test 1. These two numerical values are interpreted as the co-ordinates 
of a point which is plotted in the diagram 1a. This point is the terminus of 
the test vector 1. The test is now represented by a test vector. Each of the 
eight tests in the factor matrix F is represented in a similar manner in the 
diagram at 1a, and we then have as many test vectors in the diagram as 
there are rows in the factor matrix 7. In addition, we have the two unit 
reference vectors A and B, which represent the factors that are shown by 
the columns of F. The combination of the test vectors and the reference vectors 
for a test battery is called a factorial structure.* The factorial structure for 
the present illustrative example is shown in the diagram at Та. 

In the factor matrix there are four tests, namely, 1, 3, 4, 7, which have 
non-vanishing entries in both columns of F. These are represented in the 
middle of the first quadrant of Figure 1a. Two of the tests—2 and 5—have 


* These definitions of structure and related concepts are the same as in the Writer's 
original publications on multiple-factor analysis (1931—35). 
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non-vanishing entries only in column A. These two tests have corresponding 
test vectors that are collinear with the reference vector A. Two other tests 
—6 and 8—have non-vanishing entries only in column В of F. These are 
represented in the corresponding structure by test vectors that are collinear 
with the reference vector B. 

The geometrical representation of the test battery can be generalized in 
the statement that each factor loading ajm in F is represented as a projection 
of the test vector j on the reference axis m. The first test vector has a projection 
of .70 on the reference vector A and a projection of .30 on the reference vector 
B. The second test vector has a projection of .90 on the reference vector A 
and zero on the reference vector B. This means that the second test vector 
is orthogonal to the reference vector B. If a factor loading ajm in F is zero, 


I 


Ga 


FIGURE 2 


the geometrical interpretation is that the corresponding test vector j is orthogonal 
to the reference vector m. 


Correlations as scalar products 

'l'he intercorrelations ean also be represented geometrically in a very use- 
ful way. Each of the tests is represented in Figure 1a as a test vector. It 
will be shown that the correlation between any two tests is the scalar prod- 
uet of their test vectors. The scalar product of two vectors j and k is 
hjhx cos фу, where hj and АЛ denote the lengths of the vectors and $;; is their 
angular separation. 

In Figure 2 we have represented two test vectors j and k, whose lengths 
are h; and ль, respectively. Adjacent to the figure is also the factor matrix 
for the two tests j and k in their own plane with orthogonal axes I and II. 
The factor loadings, ал and ajs, are the projections of the test j on the two 
orthogonal axes, as shown in the figure, and a similar interpretation applies 


to the factor loadings, аһ and az: of test k. 
The correlation rj; between the two tests can be determined by the prod- 
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uct of the factor matrix and its transpose. The intercorrelation is then the 
cross product of the two rows of the factor matrix, namely, 
(1) Tj = Ajar + араз. 


Dividing by ЛЛ, we obtain 


ý cx ER 
but 

(3) i РУЙ 

(4) зы re 

©) de = sin dj, 

(6) а 


= = sin фи 
hy 4 


where фу and $;, are the angular separations between the test vectors j and 
k from the first reference axis. We have, therefore, 


Tik Р. Е 
(7) RR = соз фуд COS фи + sin фл sin фи. 
ih 


The angle фу can be expressed as the difference between фл and фи, SO 
that 


(8) dir = фл — фа; 


but the right-hand member of equation (7) is the expression for the cosine 
of the difference between the angles фл and фи. Hence we have 


(9) m = COS фу, 
jhk 

or 

(10) 


Tj = АА cos фу. 
The correlation between an y two tests is e 
vectors. 


An important fact about this theorem is 
axes. It should be noted th: 


qual to the scalar product of their test 


its independence of the reference 
ut equation (10) does not involve the reference 


GEOMETRICAL MODELS 91 


axes. Since the scalar product is determined only by the lengths of the vec- 
tors and by their angular separation, the scalar product of a pair of vectors is 
invariant under rotation of the reference frame. We can now make an inference 
which involves one of the central ideas in factorial theory, namely, that the 
correlation matrix determines the test vectors as to their lengths and angular re- 
lations, but the correlation matrix does not define the location of the reference 
axes. 

With the scalar-product interpretation of the correlation coefficient, it 
follows that a negative coefficient is always represented by an obtuse an- 
gular separation of the test vectors, the lengths A; and A; being taken as 
positive. Also, with non-vanishing lengths of the vectors, a zero correlation 
is represented geometrically by aright angle. A pair of tests with zero cor- 
relation have orthogonal test vectors. 

If two vectors are each of unit length, then the numerical values of h; 
and A, are both unity, and their scalar product is the cosine of their angular 
separation. But cos фу» is also the projection of either unit vector on the 
other. Hence we have the theorem that the scalar product of a pair of unit 
vectors is equal to the projection of either vector on the other. 

If the entries in the correlation matrix are all positive, as is the case with 
tests of the cognitive or intellective functions, then all pairs of test vectors 
are separated by acute angles. It follows that if all the correlations are 
positive, the configuration of test vectors lies inside a cone of 45° generating 
angle. 


The configuration 


The correlation matrix of Figure 1e contains all the correlations for the 
eight-test battery. Since only two common factors are involved in these cor- 
relations, the rank is 2, and it is possible to represent the test relations by a 
set of eight test vectors in two dimensions. The lengths of the test vectors 
are determined by the diagonal entries in the correlation matrix. Each diag- 
onal entry h? is the communality of a test j ,and the length of the correspond- 
ing test vector is л;. The angular separations between the test vectors are de- 
termined by the side entries of the correlational matrix in such a way that 
the scalar products of all pairs of veetors are equal to the corresponding cor- 
relations. The number of the dimensions that are required to represent the 
given correlations is the same as the rank of R, and this is the same as the 
number of common factors in the test battery. 

If we consider only the arrangement of the test vectors among them- 
selves as determined by the correlation coefficients and without regard to 
any reference frame, we call the arrangement or grouping a configuration of 
test vectors. The configuration of test vectors for the present example is 
shown in Figure 1d. The configuration at /d is identical with that at /a. 
The only difference is that the configuration and the reference aves are com- 
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bined to form a structure at fa, whereas the configuration alone is shown at 
1d without any reference frame. The configuration has been drawn inten- 
tionally in different positions at /a and 1d. 

"There is an exact relation between the test configuration at /d and the 
correlation matrix at 1e. A model or diagram of the configuration сап be 
constructed from the correlations, and the reverse is also possible, in that 
the correlations can be computed from the configuration. We lose no infor- 
mation in going from the correlations to the configuration or vice versa be- 
cause they are merely two different ways of stating the same facts. It should 
be noted especially that neither the 
reference frame. 

Just as there is a direct correspondence 
and the fest configuration, neith 


correlations nor the configuration has any 


between the correlation matrix 
er of which involves a reference frame, so 
there is a direct correspondence between the factor matriz and the factorial 
structure, both of which do involve a. reference frame. The pairs have been 
placed adjacent to each other in Figure 1 to emphasize these relations. The 
test structure cannot be defined until the reference axes have been located 
in the configuration, as shown in Figure 1а. In the same manner, the factor 
matrix at 1b cannot be defined except in terms of a particular set of refer- 
ence axes which are implied by the two columns of that matrix. 

A factor problem usually starts with a table of intercorrelations, as shown 
in Figure te. If it were determined that the rank of that matrix is 2, then we 
should know that the intercorrelations of the eight tests are produced by two 
common factors even before we could have any idea what these two factors 
are. We would also know that the configuration of test vectors would be con- 
tained within two dimensions, so that they could be represented as a plane 
diagram, as shown at 1d. When the correlational matrix is given, including 
the diagonal entries, then the configuration can be drawn. It should be espe- 
cially noted that the correlation matrix and the 
reference axes. 

The object of a factorial problem is to write a factor matrix / that will re- 
produce the correlations R by the fundamental theorem FF’ = fe. This prob- 
lem can be represented geometrically. It consists in determining the rank 
of R, which is also the number of dimensions in the test configuration, and 
the rank is also the number of columns of the factor matrix F. The con- 
figuration can then be drawn uniquely. A set of reference axes can then be 
Inserted into the configuration to form the structure at la. When the ref- 
erence axes have been inserted into the configuration, it is possible to de- 
termine the projection of each test vector on eac 
These projections constitute the ce 
should be el ear, then, that the fae 
of reference axes has been insertec 
by the intercorrelations. 


test configuration have no 


h one of the reference axes. 
ll entries аы of the factor matrix F. It 
tor matrix cannot be written until a set 
l into the configuration which is defined 
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Certain features of the test configuration can be inferred by mere inspec- 
tion of the correlation coefficients. If all the correlations are positive or 
zero, we can infer that all the test vectors are separated by acute angles. 
The largest angular separation would then be a right angle. It follows that 
the whole test configuration must be contained within a cone of 45° gen- 
crating angle when the correlation matrix has no negative entries. This is 
the situation with psychological tests of the cognitive or intellective func- 
tions, which seem never to show any significant negative correlations. 


The arbitrary reference frame 

Since the object of factor analysis is to find a factor matrix F from a 
given correlation matrix 2, so that the fundamental factor theorem FF’ = R 
is satisfied, it is of interest to note that every correlation matrix can be fac- 
tored into a factor matrix. In fact, the correlation matrix is, by definition, a 
product of two score matrices, as has already been shown in (20-ii), 


(20-ii) m = 


Since the correlation matrix is obtained computationally as a product of two 
matrices, it is evident that every correlation matrix ean be factored. We 
have, therefore, the theorem that for every correlation matrix R there exists a 
corresponding factorial matrix F such that FI" = R. 

We have seen that a given correlation matrix can be interpreted as show- 
ing the scalar products of pairs of test vectors. We have also seen that the 
factor matrix can be interpreted as showing the projections of each test 
vector on each of the co-ordinate axes. The numerical values of the projec- 
tions of the test vectors are, of course, dependent on where we put the fac- 
torial reference frame. Hence we have the important theorem that an in- 


finite number of factor matrices F can be written which reproduce any given cor- 


relation matrix R. This theorem makes it evident that when we have fac- 
{огей a given correlation matrix FR into a factor matrix F we do not have a 
unique solution. Further restrictions must be imposed on the solution P, 
so as to make it in some sense unique before we can hope to interpret the 
test coefficients. On this question there has been much controversy, which 
we shall have occasion to consider in a later chapter. 

In order to demonstrate the arbitrary location of the reference frame, we 
have drawn in Figure 3 several positions of the co-ordinate axes for one test 
configuration. The configuration of Figure 14 is reproduced in three differ- 
ent positions in Figure 3 as regards the co-ordinate axes. Below each dia- 
gram in the figure we have the corresponding factor matrix. Although the 
numerical values are different in these three factor matrices, they all repro- 

s, namely, those of Figure 1e. 


duce the same correlations, n: 1 
Since the correlation matrix R consists of scalar products of test vectors, 
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while the factor matrix F shows the projections of the test vectors on the co- 
ordinate axes, we have several other theorems that are useful in under- 
standing the relations between these matrices, X and F. Consider the test 
vector k and the unit vector X in Figure 4. The scalar product of these two 
vectors is hihi; cos rz, Which becomes h; cos $;., since X is a unit vector. 
Let Z be a unit vector collinear with X. Then the projection of Z on X is 
cos rz. But the projection of k on X is proportional to the length A; of k. 
Hence the projection of k on X is h; cos s+. We then have the theorem that 


I 


I II 1 п 


= 
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Picture З 


the projection of a test vector on any given axis is the scalar product of the test 
vector and a unit vector collinear with the axis. 


Consider a set of test vectors k 
seen that the projection of a test ve 
as hi. cos фу, 
Аһ cos фу. 
vector j is 


(11) Раг Tjik 


and a particular test vector j. We have 
› ctor kon X is h; cos фу. This is the same 
since rz and фу are the same angle. But the correlation r 


„= 
Hence the projection of the test vector k f 


on a particular test 
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Hence the correlations between a set of tests k and a particular test j are propor- 
tional to the projections of the test vectors k on an axis collinear with the test vec- 
tor j. This theorem can be applied in the geometrical interpretation of a 
column, or row, j, of the correlation matrix. Such a row shows the correla- 
tions between a set of tests, k, and a particular test j. The configuration of 
test vectors corresponding to the correlation matrix satisfies the theorem 
that the correlations in any particular row or column, j, of the correlation ma- 
trix are proportional to the projections of the test vectors on an axis through the 
test vector j. This is merely a restatement of the previous theorem. The 
proportionality factor is 1/A;, so that, if the length of a particular test vec- 
tor j is known or assumed, then one column of the factor matrix can be im- 
mediately written by (11) if we decide to locate one of the co-ordinate axes 


FIGURE 4 


through one of the test vectors j. The projections Pz; are then, in fact, the 
entries of one of the columns of the factor matrix. 

The same theorem can be restated in still another form that is important 
for factorial analysis, namely, that, for any given correlation matrix with an 
implied common-factor space, there exists in that space an axis through the 
origin on which the test vectors have projections that are proportional to the 
correlations of any given column, or row, of the correlation matrix. If the cor- 


relation matrix is of order n, there exist n such axes in its common-factor 


space. 
If the co-ordinate axes are located elsewhere in the common-factor space, 


the projections of the test vectors on these co-ordinate axes will be linear 
combinations of the projections on the axes through the test vectors. We 
infer a fundamental theorem in factorial theory, namely, that every column 
of factor loadings in the factorial matrix is a linear combination of columns of 
the correlational matrix. We shall make use of this theorem in designing dif- 


ferent methods of factoring the correlational matrix. 
It should be clear that any interpretation of the factor matrix must be 
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made in relation to the principles that are chosen for locating the 
axes in the configuration defined by the correlations. In general, the numeri- 
cal entries of the factor matrix are as arbitrary as the location of the refer- 
ence frame. It is only to the extent that the reference frame can be in some 
sense unique that the numerical entries of the factor matrix can be given any 
scientific interpretation. 


co-ordinate 


Sign reversals in the factor matrix 


When a reference frame has been inserted into the 
projections of the test vectors computed as shown ir 
should be noted that the sense in which a co 
Figure 1а is taken determines the signs of tl 
1b. If the direction of the co-ordinate vector A is reversed, then all the 
signs in column A of the factor matrix are also reversed. If the reference 
vector A represented some fundamental human trait, such as cheerfulness, 
then the reversal of direction of the reference vector would represent the 
direct opposite, such as grouchiness. Since the scalar products of the test 
vectors are independent of the location of the reference frame, we have the 
computationally useful theorem that the signs of all the entries in a column 
of the factor matrix may be changed without altering the corresponding correla- 
tion matriz. Computationally, this theorem becomes evident also if we con- 
sider the correlation coefficient as a cross product of a pair of rows of the 
factor matrix, such as Figure 1b. If we reverse the signs in both aj, and 
Gm for a fixed column m, their product remains unchanged in sign so that 
the correlation coefficient "jx remains unchanged. The psychological inter- 
pretation of this reversal in direction of a reference vector can be shown in 
terms of the example given. The correlation between two traits remains 
unaffected by the arbitrary decision to call one of the component reference 
traits “plus cheerfulness” or "minus grouchiness.” 

Now consider the reversal of all the signs in a row j of the factor matrix. 
This has the effect of reversing the direction of the corresponding test vector 
j. Its scalar remains the same, while its angular separation фу from any 
other test vector k is changed to the supplement of Фу. Hence the absolute 
values of the correlations of this test with the other tests remain unaltered, 
but their signs are reversed. The psychological interpretation can be shown 
by an example, namely, that if one variable correlates positively with “plus 
tactfulness,” then it will correlate negatively with “minus tactfulness,"' 
which might be defined as "plus tactlessness,”” 


another useful theorem that 
versed in a row of the factor ma 


configuration and the 
1 Figure 1a and 1b, it 
-ordinate vector such as A in 
ıe projections in column A of 
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signs are reversed in the row j and also in the column j of the correlation 
matrix. These two sign changes in the diagonal cell keep the sign of the self- 
correlation unaltered. This theorem can also be inferred from the factor 
matrix. If all the signs in a row of that matrix are reversed, then all the 
terms in a cross product of row j with any other row Ё are similarly reversed 
in sign so that the correlation rj; is reversed in sign except the self-correla- 
tion of test j, which remains the sum of the squares of the entries in the 
row j. We can also infer from these relations that the self-correlation of a 
test remains positive for all possible reversals of sign of tests and factors. 


The restriction that r be less than n 

In the illustrative example of Figure 1 we have eight tests and two com- 
mon factors, so that n —8 and r=2. There are certain inequalities relating r 
and n which must be satisfied in order that a determinate factor solution 
shall be possible. To introduce these restrictions, consider the case in which 
rn. This case is illustrated in Figure 2, which shows a factor pattern of 


ABCDE 


FIGURE 5 


five common factors and three tests. The crosses represent non-vanishing 
factor loadings in the factor matrix. Assume that the five common factors 
are uncorrelated. The correlations between the three tests determine the 
correlation matrix, which is of order 3. The rank of that matrix cannot ex- 
ceed 3, but we have set up the problem to represent more than three com- 
mon factors. If such a correlation matrix were factored, it would give a fac- 
tor matrix of not more than three columns or factors. The numerical entries 
in such a factor matrix would determine the correlations exactly, but the 
three factors would probably be artifacts as far as any scientific problem is 
concerned. The relations might have some statistical interest, and they 
might be useful for constructing various empirical prediction formulae, but 
they would not reveal the five factors that were operative in the test vari- 
ances. Factor analysis is built on the postulate that the number of common 
factors does not exceed the number of tests in the battery. As far as a numerical 
solution is concerned, every correlation matrix can be factored routinely if 
we are willing to accept as many factors as there are tests. The scientific 
interpretation and validity of the factors is another matter. 

The geometrical interpretation of the postulate that тл can be seen if 


98 MULTIPLE-FACTOR ANALYSIS 


we consider the configuration of » test vectors, If these vectors are defined 
in terms of their components in more than n dimensions, the n vectors can 
also be described in an n-dimensional subspace which is spanned by the 
n vectors. That is the space defined by the factor matrix, from which it 
would not be possible to identify the larger number of factors that produced 
the test correlations. 

These considerations raise the question of how one can be sure that any 
particular battery of » variables really does involve | 
factors. There is no guarantee 


ess than n common 
beforehand that an y proposed battery of n meas- 
urements will involve less than n common factors that have significant interpre- 
tation. To produce such a situation is as much dependent on scientific in- 
tuition as other scientific experiments are subject to intuition for Success 
and significance. One can, of course, factor any given correlation matrix to 
determineits rank. If the rank of the correlation matrix turns out to be com- 
parable with its order, one has merely made a bad guess, and one must start 
over again with a revised test battery in the same domain. 


The test space and the common-factor space 

The relation between the correlation matrix and the configuration is very 
markedly dependent on whether we insert unity or the communalities in the 
diagonal cells of the correlation matrix. The principles involved here can 
be illustrated in terms of the correlation between two tests, ў and k. The 
correlation matrix would be of order 2. If we write unity in the diagonal 
cells, there will be only one correlation coefficient to determine, namely, rjr 
The corresponding configuration is a pair of unit vectors, which are sepa- 
rated by an angle whose cosine is the given correlation coefficient. If we con- 
sider the tests of a battery as represented by unit vectors to cover the total 
variance of each test, then we have a configuration of n unit vectors in a 
space of (n+r) dimensions, which represent the common factors and the 
unique factors. Such a Space is called the test space to distinguish it from the 
Space that is defined by the common factors, which is called the common- 
factor space. Since the test Space represents usually more factors or param- 
eters than there are tests, we shall not use it except in special problems 


where the total test space is indicated. The total test space is of n dimen- 
Sions if we disregard the factorial composition of the tests. 


Summary of geometrical concepts 


The geometrical represent 
rized here, These theorems 
lems with any determinate 


ations described in this chapter will be summa- 
and definitions are ай] applicable to factor prob- 
number of factors or dimensions, 

The factor matrix and the configuration 


1) A reference frame must be inserted її 


3 ato the configuration before any 
factor matrix can be written, F 
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2) Since the co-ordinate axes are not determined by the experimentally 
obtained correlation matrix, every factor matrix is in the nature of a fac- 


torial interpretation of the given correlations. 
3) Each row of the factor matrix can be interpreted as showing the rec- 


tangular components of a test vector. 
4) Each column of the factor matrix identifies a factorial axis in the 


structure. 
5) Each reference factor is represented by a unit reference vector. 
are plotted pairs of columns of the factor matrix 


6) The graphs on which 
and the refer- 


show the factorial structure which combines the test vectors 


ence frame. 
7) If the components of an orthogonal factor matrix are all positive or 


zero, the structure has all test vectors in the first quadrant or octant. 


The correlation matrix and the configuration 


8) The correlation matrix can be interpreted as showing the scalar prod- 


ucts of all pairs of test vectors. 
9) The length of each test v 
its communality. 
10) The correlation m 
unique configuration of test v 


model if the rank of the matrix is less than 4. 
11) The correlation matrix does not determine any reference frame. 


12) If the dimensionality or rank is less than 4, a physical model can be 
constructed to show the configuration alone or the factorial structure which 


ineludes the configuration and the reference axes. 
13) No information is lost in representing the correlation matrix as a 


configuration, or vice versa. — x N 
14) A correlation matrix with coefficients that are all positive or zero 
determines uniquely а configuration of test vectors in which there are no 
obtuse angular separations. 
15) The scalar product of 
axes, i | | 
16) The scalar product of a pair of unit vectors is equal to the projection 

of either vector on the other. | 
presents a pair of test vectors with 


17) A positive correlation coefficient re | 
acute angular separation. А negative correlation represents a pair of test 


veetors with obtuse angular separation. x e 
18) The projection of a test vector on any given axis is the scalar product 


of the test vector and a unit vector collinear with the axis. 
19) The correlations between a set of tests k and a particular test jare 


proportional to the projections of the set of test vectors Ё on an axis collinear 


with the test vector J. 


ector is equal to the positive square root of 


atrix with known diagonal entries determines a 
ectors that can be represented as a physical 


a pair of vectors is invariant under rotation of 


100 MULTIPLE-FACTOR ANALYSIS 


20) Given a correlation matrix with an implied common-factor space, 
there exists in that space an axis through the origin on which the test vectors 
have projections that are proportional to the correlations of any column, 
or row, of the correlation matrix. 

21) Every column of factor loadings in the factor matrix is a linear com- 
bination of columns of the correlation matrix. 


Sign reversals in the factor matrix 


22) The signs of all the entries in a column of the factor matrix may be 
reversed without altering the corresponding correlation matrix. 

23) Reversing the signs in a column of the factor matrix reverses the di- 
rection of the corresponding reference factor and reference vector. 

24) If all the signs are reversed in a row of the factor matrix, then all the 
signs are reversed in the corresponding row and in the corresponding column 
of the correlation matrix, 

25) To reverse the signs in a row of the factor matrix reverses the direc- 
tion of the corresponding test vector, 

26) The self-correlation{of a test remains positive for all possible reversals 
of sign of tests and factors, 


CHAPTER IV 
A FACTOR PROBLEM IN TWO DIMENSIONS 


Introduction 

So far we have considered the assumptions of factor analy 
mental equations, and some of the geometrical concepts of factor theory. 
In discussing the sections of Figure 1 in chapter iii we have seen the rela- 
tions between the factor matrix, the factorial structure, the test configura- 
tion, and the correlation matrix. We shall now consider the illustrative 
problem of that chapter as if it were a factor problem to be solved. The 
problem starts with the given correlation matrix, which is ordinarily de- 
termined from experimental data. The problem now is to analyze the cor- 
as to diseover the factor matrix Ё and the structure as 


s, the funda- 


relation matrix so 
shown. The correlation matrix of the previous chapter will be used now as 
the given data, and we shall determine from it the factor matrix. It is a 
simple matter to construct a correlation matrix from a given factor matrix, 
but it is usually more difficult to extract the factor matrix from a given cor- 
ation matrix. It is the latter which constitutes factor analysis, where the 


rel 
al data are in the form of correlation coefficients from which a 


experiment 
factor matrix is to be found. 

The solution of a factorial problem consists usually of two steps, namely, 
(1) to factor the given correlations into a factor matrix with arbitrary ref- 
erence frame and (2) to rotate the arbitrary reference frame into a pre- 
ferred or simplifying position. The first is known as the factoring problem, 
and the second is known as the rotational problem. We shall describe both 
steps for the present example. There are several methods of factoring a cor- 
relation matrix, and there are several methods of solving the rotational 

In fact, most of the theory and computational methods of factor 
are concerned with these two main problems. Here we shall start 
factoring methods, namely, the diagonal 


problem. 
analysis 
with perhaps the simplest of the 


method. 


The diagonal method of factoring 

Before proceeding to the factorial solution of the correlation matrix of 
the previous chapter, we shall describe the diagonal method of factoring 
by a small five-variable matrix in Table 1. This method of factoring is not 
well suited to large problems involving fallible data, but it is a simple rou- 
tine method that can be used on a correlation matrix of any order and 
any rank. 1 

The correlation matrix of Table 1 will be used to illustrate the diagonal 
method of factoring. The total number of independent correlation coeffi- 
101 
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cients is }n(n+1) if the diagonal entries are included. Since the correla- 
tion matrix of Table 1 is of order 5, there are fifteen correlation coefficients 


Table 1 


Correlation Matriz 


1 2 3 4 5 

1 | .85 .98 80: 18 

2:14. 296 80 16 56 

3 .36 40 34 53 

4 .82 16 97 27 

5 .18 .56 27 .58 
R 


available for determining the factorial matrix. 

The corresponding factor matrix is shown in literal form in Table 2 and in 
numerical form in Table 3. In Table 2 all the entries above the principal 
Table 2 


Factor Matrix 


Table 3 


Factor Matriz 


| I II III IV У 


1 ап 0 0 1 0 0 0 
2 аз 0 0 2 0 0 0 
3 аз 0 0 3 J58S 0 0 
k ay an 0 4 - .403 .000 0 
5 ам ам ам 5 -595 .433 .000 .000 


Р r 
diagonal are zeros, This circumstance w 
ly, geometrically in terms of the arbitrary location of a reference frame and 
algebraically in terms of the correlations expressed as cross products of 
rows of F. It should be noted that the number of coefficients to be deter- 
mined in Table 2 is exactly equal to the number of correlation coefficients 
in Table 1. It will be shown that each correlation coefficient in R determines 
one of the factor loadings in Ё, so that the method can be extended to the 
case where the rank of R is equal to its order, in which case the number of 
factors in F is equal to the number of variables in R. When the rank of R 
is less than its order, it will be found that a smaller number of columns will 
be required in F, In the present. example the rank of R is 3, and hence the 
last two columns of Table 3 contain only zeros. 


The zero entries in Table 2 can be explained in terms of the arbitrary 
reference frame. Let the first axis be collinear with the first test vector. 


ill be explained in two Ways, name- 
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Then this vector will have a projection on the first axis equal to the length 
of the vector, namely, the square root of its communality. The test vector 
1 all other possible axes that are orthogonal to 
the first axis. Hence the remaining entries in the first row of F are all zeros. 

The second axis may be placed orthogonal to the first axis and in the 
plane that is spanned by the first two test vectors. Then the second test 
vector will have projections on the first two axes but not on any others. 
Hence all but the first two entries in the second row of F will be zeros. The 
third axis may be placed in the space spanned by the first three test vec- 
tors and orthogonal to the first two axes. The third test vector will then 
first three axes but not on any of the other axes. 
We are assuming that the successive tests that are used for the location of 
are so chosen by inspection that they are linearly inde- 
doing this will be shown in the numerical ex- 
ed orthogonal to the first three axes and in 
the four-dimensional subspace that is spanned by the first four test vectors. 
Hence the fourth test vector will have projections on the first four orthogo- 
nal axes but not on any of the other axes. Т his process is continued until a 
sufficient number of dimensions has been defined to ancou for the a 
products of the correlation matrix. This location of the reference pne 
produces a set of zero entries in the upper half of the cwn matrix, et 
shown in Table 2. The zero entries of this table can be inserted at the start 


of the factoring. 
The triangular arran 


will have zero projections or 


have projections on the 


the reference frame 
dependent. The manner of 
ample. The fourth axis 15 plac 


gement of zeros in Table 2 can also be explained 
algebraically in terms of the cross products of F that ppa e the 
correlations in X. The sum of the squares of Hoan row o able zi 
equal to the communality of test 1. Since there is only one entry in the 


first row of F, we have 


‚„ "Table : 

à is is recorded in Table 3. 

л sina PESE of the first two rows of F must reproduce the correla- 
ле cross produc 


tion ra,=.28. Hence we have 


апам = 38, 
922an = .28, 
9045 


The correlation between the first test and any other test 7 becomes 


andj = Ti, 


(1) 
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or 


я 
(2) ау = Hl 


The entries in the first column of Table 3 are therefore proportional to the 
first column of correlation coefficients in R, as shown in equation (2). 

When the first column of the factor matrix has been recorded, we turn to 
the communality of the second test, which is the sum of the squares of the 
entries in the second row of F, Hence 


az + a, = 80, 


in which as» is the only unknown. 
recorded as .841 in Table 3. 

The correlations of the 
in the form 


It can therefore be determined, and it ix 
second test with any other test j can be written 


аза + азза; = 772, 


in which aj; is the only unknown. The second column of Table 3 is thus 
numerically determined. 


The communality of the third test is, similarly, 


3 
2 . 
У Gn = 133 , 
m 


in which a5; is the only unknown, 


It is recorded as .588 in Table 8. 
The correlation of test 3 with 


any other test j can be written 
3 


зһ р = Py, 


m=] 


in which ау is the only unknown. Hence the third column of Table 3 be- 


comes numerically determined. 


In this numerical example, the rank of 7? is 3, and hence, if this process 
is continued, the remaining factor loadings in Table 3 will be found to van- 
ish. Three columns of F represent three arbitrary reference factors, which 
account for the correlations in R. 

In the diagonal method of factoring, the selection of each successive test 
Should be made on the basis of the residual variances of the tests. For ex- 
ample, when two columns of Table 3 have been numerically determined, 
one might compute the sum of the squares of each row of F and compare 
these sums with the communalities as shown in R. Th 
for determining the third axis, which shows the ] 


but any test can be used as a pivot for the next 
ciable diagonal entry. 


at test might be taken 
argest residual variance; 
factor if it has an appre- 
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The practical limitation of the diagonal method of factoring for fallible 
data is that the method pivots on one test at a time. The choice of a diagonal 
entry which is estimated with practical data determines the factor loadings 
in a manner which is very sensitive to errors in the estimations for the 
diagonal cells. In general, computing methods are preferred in which all 
the data contribute to the determination of each parameter by summations. 
However, the diagonal method is theoretically correct, and it can be used 
as a routine computational method of factoring where the given correla- 
tional entries can be regarded as exact. 


A two-dimensional example 
The correlation matrix in Figure 1-iii has been reproduced here as 
Table 4, and it will be factored by the diagonal method. We choose arbi- 
Table 4 


Correlation Matrix 


1 2 3 4 5 6 ré 8 
1 +58.  .03  .46 „57 д  .18  .45  .24 
2 .63  .8l .90 .54 .45 .00 .27 .00 
3 46 .86  .52 .42 .20 ..36 .60 .48 
4 51 4  .42  .40 .30 .18 .42 м 
5 350 .45  .20 80. 225 О  .15  .00 
6 18 .00 .36 .18 .00 .36 .48 .48 
7 45. AUT „60. .42 15 з 155  .04 
8 .24  .00 .48  .24 .00 .48 .64 .64 
> 3.40 3.06 3.40 3.06 1.70 2.04 3.74 2.72 


trarily the first test to determine the first reference axis. The communality 
of test 1 is .58, and hence the desired multiplier is 1 'М/.58 = 1.313064. 
Applying this stretching factor on the first column of correlations in Table 4, 
we obtain the first column of factor loadings which are entered in Table 8. 

The computational checks have been included in this example. First, we 
sum the columns of Table 4, as shown in row E. The sum for the first col- 
umn, 3.40, is treated as an additional entry in this column, Applying the 
stretching factor to it, as for the correlation coefficients in the first column, 
we have 4.464, which is recorded in Table 8, row Ch. The actual sum of 
that column of Table 8 is 4.465. Since the predicted check sum (Ch) agrees 
with the actual sum (х) in Table 8, we assume that the computations and 
the copying into the table are correct. We shall not specifically mention 
sach check sum, since the principle is essentially the same for the rest of 
the computations. Almost every novice in computing minimizes the im- 
portance of formal computational checks. After he has wasted some days 
hunting for his errors, he learns that it saves both time and annoyance to 
proceed with formal explicit checks at every step in computing. In the 
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publication of computational results it is not customary to distract the 
reader with the computational checks. 

When the first column of the factor matrix in Table $ has been numerical- 
ly determined, the cross products of this column are listed as shown in 
Table 5. The summation in Table 8 is here again treated as an additional 
entry, which gives row Ch in Table 5. This table shows the contributions 
made by the first reference factor to the given correlations, 


Table 5 
First-Factor Products a лац 
3 2 3 4 5 6 7 8 
1 .460 .351 .180 .240 
2 -500 . 380 .195 261 
3 .365 .278 .143 .190 
4 .405 E .308 .158 211 
5 .278 .308 8212) .109 145 
6 .143 .158 .109 .056 .074 
7 -357  .396  .272 | 139 .186 
8 -190  .211 -145 — .074 .099 
Ch 2.697 2.992 2 1.054 2.639 1.406 
= 2.698 2.992 2 1.054 2.638 1.406 
Table 6 
Virst-Faclor Residuals Tajik = Tik — арац 
1 2 3 4 5 6 7 8 
2 | 00! 000 .000 —.001 —.001 б ой ‚000 
2 ‚000.126 — 140 —.014 .070 —.195 —.219 ~ 961 
3 .000 —.140 .155 .015 —.078 4217 .243 .290 
4 —.001 —.014 .015 .001 —.008 .022 .024 .029 
5 —.001 .070 —.078 — .008 .038 —.100 —.122 —.145 
6 .000 —.195 .217 -022 —.109 . 304 .341 
7 -000 —.219 +243 .024 —.122 ‚841 -381 
8 .000 —.261 .290 .029 —.145 .406 .454 
Ch —.003 —.633 .702 i . -986 — 1.102 1 .314 
> —.003 —.633 -702 - 986 1.102 1.314 


ч + 
" The cross products shown in Table 5 are subtracted from the correla- 
tions in Table 4. These are denoted 


(3) А — 


Which are shown in Table 6. The subscript 2 in 72.54 refers to the fact that 
these are residual correlations from which the second factor loadings are 
to be determined.* Table 6 can now be treated Just like Table 4, in order 
to determine the next column of factor loadings. 


* In some of our previous w 
factor residuals instead of first- 


jk — ала = т» 


ork we have used the notation rj; to represent second- 
factor residuals, as in this problem, 
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Column 8 in Table 6 shows the highest diagonal entry, and we therefore 
choose it as the second pivot test. The residual communality for test 8 is 
.541, so that the multiplying factor is now 1/V.541= 1.359569. Apply- 
ing this stretching factor to column 8 of Table 6, we get the second column 
of the factor matrix in Table S. 


Table 7 


Second-Faclor Products аак 


1 2 3 4 5 6 7 8 
1 000.000 .000 .000 .000 .000  .000  .000 
2 ‘000 .126 —.140 —.014 .070 —.196 —.219 —.261 
3 1000 —.140  .155 .015 —.078  .217 .943 1290 
4 ‘000 —.014 .015  .002 —.008 .022 024 1029 
5 1000 070 —.078 —.008  .039 —.109 —.122 —.145 
6 1000 —.196  .217  .022 —.109  .305 .3 .406 
7 1000 —.219  .243  .024 —.122  .341 .381 .454 
8 1000 —.261  .290 .029 —.145 .406 .454 .542 
Ch .000 —.634  .701  .070 —.352  .986 1.102 1.314 
3 1000 —.634  .702  .070 —.353  .986 1.102 1.315 
Table 8 
Faclor Matriz F "able 9 
= Given Factor Transformation Rotated Factor 
I II Matrix Matriz Matriz 
———— 
2 I п 1 B 1 B 
3 1 000 I| Aw Xe 1 |.70 .30 
5 2 —.355 П Xia dew 2 | .90 .00 
6 3 394 __ 3 |.40 .60 
2 4 039 — چ‎ de р 60) 0 
$ 5 .197 - 5 |.50 .00 
р 6 552 6 | .00 .60 
Ch 7 or у ч вт |.30 .80 
х [4.465 1.786 8 786 1 "BN © 
r2 m e: —.400 .917 ل‎ 
d .58 .541 F, 15 е n 
Vd .761577 .735527 А 
1 Vd 1.313065 1.359569 


The cross products of the second column of the factor matrix give 
(азал), Which are shown in Table 7. These are the contributions of the 
second reference factor to the correlations in Table 4. Subtracting the en- 
tries of Table 7 from the first-factor residuals тә. in Table 6, we find that 
they vanish, showing that the two factors in Table 8 are sufficient to ac- 


count for the given correlations. ii 
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Rotation in two dimensions 

The factor matrix of Table 8 has been plotted in Figure 1, where we see 
the two-dimensional configuration and the arbitrary reference axes T and П. 
In choosing a reference frame for the interpretation of the variables in terms 
of factors, we must decide on some criterion by which to make the choice. 
The first axis might be passed through the centroid of the eight points in 
Figure 1, or the first axis might be so placed as to maximize the sum of the 
squares of the projections of the eight test vectors. We might place the 
axes in such positions that the negative projections will be eliminated or 


FIGURE 1 


minimized as far as possible. 


Another criterion which can sometimes be 
used is to place the axes so 


as to minimize what we have called the “сош- 
plexities" of the tests. When this criterion is applicable, the axes are so 
placed that a large number of zero entries appear in the factor matrix. The 
rotational problem is one of the most important in factor analysis. 

In the present example we see in Figure 1 that the angular span of the 
configuration is a right angle, so that a frame can be chosen which eliminates 
all negative factor loadings and which reduces the complexities of four 
tests to one instead of two. The two axes chosen by these criteria are de- 
noted A and B in the figure. 

The numerical problem now is to write 


a factor matrix which will repre- 
sent the two orthogonal factors 


A and B instead of the arbitrary orthogonal 
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factors J and ГГ, which appeared in the factoring of the correlation matrix. 
In Table 9 we have the factor matrix F, for the axes J and IIJ (from Table 8), 
and we want to find an orthogonal transformation matrix A so that F,A— 
Ёз, where Fa will be a factor matrix for the axes A and B.* The notation 
Fy, Fo, Fa, ... , will henceforth be used to represent successive rotations 
of the reference frame. 

Since the present configuration of test vectors is contained in two dimen- 
sions, the orthogonal transformation A will be of order 2X2. It will be 
useful to make а geometrical interpretation of the cell entries in the matrix 
A of the transformation. The cell entries in the first column A of this matrix 
are denoted Xi, and Asa. These are the direction cosines of the unit vector A 
referred to the frame I-II. Another way of saying the same idea is that the 
unit vector A in Figure 1 can be expressed as a vector sum of its projections 
on the axes J and 77. We have, then, for the vector 4, 


(4) А = Nal + MI. 


The first column A of the orthogonal matrix A in Table 9 states that the 
vector A is Ma of the unit vector J plus №, of the unit vector IZ. In Figure 1 
these coefficients of equation (4) can be determined graphically. Since A is 
a unit vector, we shall expect that 


(5) M+ М = 1. 

By inspection of Figure /, and normalizing, we have 
(6) А = .9171— 40011, 
and these coefficients are entered in the numerical form of the transforma- 


tion matrix A in Table 9. 
A similar interpretation applies to the second column B of the transfor- 


mation matrix. From the figure we obtain close estimates of the projec- 
tions of B on J and IJ, namely, № and №, which, when normalized, give 


the unit vector B as the vector sum, 


(7) B= 4007 + 91711. 
e noted that the rows of the transformation matrix 


Table 9 it should b 
eee T, while the columns refer to the axes A and B. If 


refer to the axes J and 7 hile th 
we perform the matrix multiplication 


(8) PA = Г, 


* For theorems related to rotation of axes see “Mathematical Introduction" and 


theorems in chap. iii. 
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the product is of order n X2, and the columns are denoted A and B. This 
has been done in Table 9, where F» is the desired factor matrix for the fac- 
torial axes A and B. This matrix is identical with the factor matrix with 
which we started in chapter iii. 

In writing this factorial solution, we might have labeled the two axes A 
and B in reverse order in the figure, or we might have reversed the order of 
the two columns of the transformation matrix A. The effect would have 
been merely that the columns of Fa would appear in a different order. It 
should be evident that the order in which the columns of Fe are written 
would have no effect on the interpretation of the factors in terms of the tests. 

The extension of the reasoning illustrated here to three or more dimen- 
sions involves essentially no new ideas, although the higher-dimensional 
problems are more complex. In problems involving more common factors 
it is more difficult, for example, to identify the nature of the configuration, 
which is obvious in a single diagram in two dimensions, 


CHAPTER V 
THE GROUPING METHOD OF FACTORING 


The diagonal method of factoring a correlation matrix has been described 
in chapter iv. While that method is simple in principle, it has a serious dis- 
advantage when dealing with fallible data, namely, that it pivots succes- 
sively on single tests and on single correlation coefficients. If any of these 
individual coefficients is in error, the calculations that are based on them 
are similarly in error, so that the residuals become larger than when some 
summational procedure is used. For this reason the centroid method was 
developed. The grouping method to be described here is a variant of the 


centroid method. 


MEN 


| [7] 


Figure 1 


Consider the correlation matrix rj, in Figure 1. Instead of summing 
over all the variables, which is the starting-point for the centroid method, 
we shall sum here over a group of tests. The group should contain at least 
three or four tests. The successive groups to be used should be so selected 
that the intercorrelations of the tests within one group are large, relative 
to the magnitudes of the coefficients in the table. Detailed routine for 
selecting the groups will be shown in the numerical example. Let there be s 
tests in the selected group. Let the rows of the selected group be denoted 
by the subscript и, and let the corresponding columns be denoted by the 
subscript v. Let the rank of the correlation matrix be r. 

Let us augment the correlation matrix by a row, as shown in Figure 1, 
of which each element, fr, is the sum of the coefficients in the u-rows for 
column k. These elements are р 
th = У", 

ad 


u= 


"uk y 


(1) 
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as well as the sum T. Applying the multiplying factor m, we have the bot- 
tom row of the table, which shows the first-factor loadings. These are copied 
as the first column of the factor matrix in Table 6. | 

In Table 4 are shown the first-factor residuals. These are (rjj — ала). 
These entries are obtained by subtracting elements of Table 3 from the 
corresponding elements of Table 2. 


T'able 2 


Correlation Matriz 


x x x 
1 2 3 4 5 6 1 8 
.58 46 51 .24 
2 .63 .36 .54 .00 
Ks .46 52 „42 55 
4 51 .42  .45 2- 
5 .85 20  .30 00 
6 18 36  .18 -48 
x7 45 .60 м7 64 
х8 .24 48 24 -64 
t | T.15 .63 1.60 1.08 1.76 5.33 =7 
I 498 .273 .693 .468 762.4331 =m 


Table 3 


First-Factor Products 


1 2 3 4 5 6 7 8 
-076 
> .041 
.324 .105 


.396 
.219 .071 .268 
-O71 .023 .087 
.268 .087 .327 
.399 .130 .488 
.357 .116 .436 .650 .581 


очо њом 


In order to compute second-factor loadings, a column is chosen which 


has a relatively large number of the higher coefficients. Test 2 is so selected, 
and its coefficients are arranged in descending order of absolute values in 
the first column of Table 5. The lower coefficients are ignored. The inter- 
correlations are recorded as shown. Test 8 has a negative correlation with 
test 2, so that test 8 is recorded here with negative sign (—8). The correla- 
tion between tests (+2) and (—8) is +.208. A similar interpretation applies 
to the other coefficients in this table. A group of tests is to be selected in 
Which the signs of the intercorrelations within the group are all positive. 
According to this criterion, all the tests in Table 5 could be included in the 
group, with tests 6 and 8 taken as negative. The coefficients of several 
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tests are rather low, so we retain only four tests for the second group, name- 
ly 2, 1, 4, 5. These are marked in rows and columns of Table 4. 

The row £; is determined by summation of four coefficients in each col- 
umn, and the value of 7 is computed as shown. Applying the multiplying 
factor m, we obtain the second-factor loadings in the bottom row, II. These 
are copied as the second column of the factor matrix of Table 6. 


Table 4 


First-Factor Residuals 


x x x x 
1 2 3 4 5 6 7 8 
х1 Tma 1277 .025 —.139 
<2 “171.412 1037 —.208 
3 040 .06 1009 — 048 
x4 096 :231 à 10921 —.117 
x5 7005 .229 .227 —.087 .020 —.116 
6 | —036 —.088 —.087 .033 —.008  .044 
7 009  .021  .020 —.008  .002 —.010 
$ _ 048 = 117 —.116 .044 —.010 .059 
t 2.050 477 1.149 1.139 —.436 .103 —.580 5.715 =T 
I1 "g8 200 481 .476—.1892 .043 —.243 — .4183— m 


Table 5 


Second Group 


L2 +1 +4 +5 -8 43 -6 
+2 wg 
+1 494 — .332 
, 4 Sy; 28 
i 400 274  .999 .227 Д 
28 208 139 -117 116 .059 
+3 171 115 .096 095 048 040 
ur. Ma ‘iog 08  .087  .044  .036 033 


Table 6 


Factor Matrix 


H п 

576 
1 ‚858 
5 .200 
| “481 
5 ‚476 
6 —.182 
7 043 

pare | 

8 .243 


116 MULTIPLE-FACTOR ANALYSIS 


When the second-factor products (аваз) are subtracted from the first- 
factor residuals in Table 4, it is found that the second-factor residuals all 
vanish. The reason for this is that the given matrix of Table 2 is of rank 2. 
The grouping method is exact in the sense that if the given correlation 
matrix is of rank т, then the extraction of r factors by this method will give 
residuals that all vanish identically. This is also true of the centroid meth- 
od to be described later. 

In Figure’3 we have plotted the factor loadings of Table 6. The 


configura- 
tion here is the same as that of Figure fa in chapter iii. The 


apparent. dif- 
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FIGURE 3 


ference is due to the interchange of abscissae and ordinates. This is asso- 
ciated with the fact that in multiple-factor analysis there is no distinction 
between independent and dependent variables. The arrangement of the 
columns in the factor matrix is of no significance, and, in plotting two- 
dimensional sections of a configuration, it is of no consequence whether 
any particular factor is plotted as abscissa or as ordinate. An orthogonal 
transformation of order 2 can be written, as previously described, by which 
Table 6 is carried back to the factor matrix of Figure 1b in chapter iii. 
The grouping method has been used on large factor studies. It is com- 
parable with the centroid method. Each has certain advantages that will 
be discussed in a later chapter. The check rows and columns w 
from the present chapter in order that the prine 
be described without the distractions of comput 


ere omitted 
iples of the method might 
ational detail, 


CHAPTER VI 
FACTORS AS EXPLANATORY CONCEPTS 
Factor analysis was develo s ientifi 
analysis was ped as a scientific method of investigati 
Tu йы à S ating re- 
Г у, measurements that are not well understood. In order to illusteata th 

ne ой we shall apply it to a problem in which the relations bubween the 
measurements are simple and well known. Instead of dealing with шал 
ап 


бепе Ан 
св and their test scores or other measurements, we shall consider а set 
of s кы ефе м т _ VB Se 
simple physical objects. In Table 1 we have listed twenty-seven суй 
у ylin- 


Table 1 


Twenty-seven Cylinders 


Суйе, Cylin- Cylin 
der „ЕУ, N а: ies bos 
1 1 Ра 1 10 1 2 2 

ag es A CENE NEC 2 2 3 2 3 
GEM Me is; T £39 2 1 3 3 
ПР iN Bm 2 3 2 o3 9 3 3 
-— is FE 3 2E Sad 
ENS E 18 th 3 25 1 0 58 
Buc So d iz: 35 de 2 20 2 4 3 
b NS FEA j 4 4 2 DT. jar “An a 


ler is completely determined by two parame- 


ders. The shape of each cylinc 
and the length l. For each cylinder is also 


ters, namely, the diameter d 
listed the specific gravity s. 

_ With these given measures We с 
Ing seven measures: 


an compute for each cylinder the follow- 


1) diameter d , rd 
х= a 

2) length 7 4 
3) base area а, c= rdl; 
4) side area с, ji rel 
Ду. 


5 volume, 
peat Р 


w= 80. 


6) diagonal t , 
7) weight w. 

Th ened ири | 
e computed values of these measures were correlated. The result is 
atrix of Table 2 for the seven measures on the 


Shown in the correlation m 
population of twenty-seven cylinders. The diagonal entries were computed 
117 


118 MULTIPLE-FACTOR ANALYSIS 


as the best-fitting values for rank 2 by methods to be described in later 
chapters. The correlation matrix of Table 2 will now be regarded as the 
starting-point for a factor analysis in which we shall try to extract as much 
information as possible about these seven measures solely from the inter- 
correlations. 

We shall consider this problem as if the actual cylinders were available, 
each one identified by its number and by its seven scores. But we shall as- 
sume that we know nothing about how each of the measurements was taken 
and that we know nothing about how the measurements happen to be cor- 
related. For the purposes of this illustrative example, this situation corre- 
sponds to that in which each member of an experimental population is 
available and is identified as to his several test scores or other measurements 
but in which we do not understand what the measurements mean or why 
the measurements happen to be correlated. 


Table 2 


Correlation Matrix 


1 2 3 4 5 6 x 
1 .000 .812  .895 ‚746 
2 1.000 541.348 .290 
3 .000 .803 — .905 ‚754 
4 .541 .968 .969 .807 
5 .348 .969 — .962 .834 
6 .823 -874 — . 767 .639 
T .290 .807 .834 .672 


Before starting the factoring, we inspect the intercorrelations that have 


been given to us for analysis. Perhaps the first thing to be noticed is that 
all the intercorrelations in this problem are positive or zero. This means 
that the configuration of test vectors does not spread over more than a 
right angle and that all the test vectors are contained within a cone of 45° 


generating angle. This is also the situation actually found experimentally 
with tests of the cognitive functions. 


Inspection of the diagonal cells shows re 
except for variable No. 7, w 
Studies one does not find co 


markably high communalities 
hose communality is 672. In most experimental 
mmunalities of .96 and higher unless the domain 
has been explored so completely that most of the variance of cach test is 
fairly well known. Variable 7, whose communality is .672, h 
thirds of its variance in common with the other tests of this ba: 
one-third of its variance is unique. In this example it is ea 
this should happen; but, in factorial studies generally, the nature of unique 
variance is usually unknown. Variable 7 is the weight of the cylinder, which 
is determined by the volume and the specific gravity. The volume is a func- 
tion of diameter d and length /, and hence the variance in this characteristic 


as only two- 
ttery. Hence 
sily seen why 
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is shared with the other measurements, which are also functions of d and I. 
But the specifie gravity s enters into only one variable of the battery, 
namely, the weight w, and hence the individual differences among the cylin- 
ders as regards specific gravity produce unique variance in w. That part 
of the variance of w which is due to volume is common-factor variance, 
while that part of the variance of w which is due to specific gravity is unique 
variance. The common-factor variance of w is its communality, while the 
unique variance is the complement of the communality. 

The correlation matrix of Table 2 was factored by the grouping method 
that was described in chapter v. Tests 1, 3, 4, 5, were used for the first group, 
and tests —1, +2, H4, +5, +6, were used for the second factor. The 
resulting factor matrix is shown in Table 3, а. This factor matrix was plotted 
in Figure 1, with the orthogonal axes J and 77 as shown. 


Table 3 


Factor Matrices 


a b " 
Ш 12 By Bs 
р 1.989 1 
2 2  .000 2 
3 3 .992 3 
1 4 .821 1 
5 5 „810 5 
6 б 1547 6 
7 7^ s 7 


With the factoring completed, we have before us the factor matrix of 
Table 3, a, and the configuration of Figure 1. We also ауе пе twenty-seven 
cylinders, each identified with its seven measures oF с. we have, 
Supposedly, no knowledge of what the клор. Шр: n «pi айе 
geometrical example we hope to show the пае па oe ai actorial 
analysis as a scientific method. If the principles вапше we M а iom case 
in which the underlying relations are definite and well о рет ша the 
application of the method to more complex pare! п m а 
With more confidence and with a better understanding of its possibilities 


and limitations. ation of the reference frame 7-77 that is 


It has been shown that the loc | sone 
obtain d ар toring is essentially arbitrary. Its location in the configura- 
ained in factoring 15 ез5 у 


і -toring. In the present case the axes 7 
tion d " the me actoring 
epends on the 


thod of fe 
and II letermined when the successive groups of tests were chosen, 
* were deter А Я Sh 

The method of choosing the groups 18 not unique. Hence we should not be 
Р i D 


Н sical i pretation of factors 
misled in assuming some simple physical interpretation of fac tors 7 and П. 
ass g Р ach group constitutes a cluster with 


They may have physical meaning if e 
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some trait in common. In the present case there is no definite cluster about 
axis Г, and there is no cluster at all near axis JT. 

The seven test vectors in Figure 1 spread in a fan from tests 1 and 3 to 
test 2, with the other test vectors between them. It can be seen by inspec- 
tion that the fan covers a right angle. This fact is associated with the fact 
that test 2 correlates zero with 1 and 3 as shown in Table 2. Tests 1 and 3 
are practically collinear, and this is associated with the fact that their corre- 
lation is practically unity. 

One alternative is to locate the reference frame at A; and А» because this 
orthogonal pair of reference vectors incloses the whole configuration. If 
we put the reference frame in this position, the factor loadings will be all 


"mc 


* 


ъ 
©, 
2 


*B 


Ficure 1 


- 
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positive or zero, as shown in the factor matrix of Table 8, b. A factor matrix 
of this type and of rank 2 is associated with the fact that all the intercorre- 
lations are positive or zero. The extension of this relation to 3 and higher 
dimensions will be discussed later. An interpretation can often be found 
for factors, such as A; and А», which inclose a whole configuration. 
Another alternative location of the reference frame is to put the first 
axis in the middle of the configuration. Such an axis can be located in the 
middle of the span between A; and As, or it can be зо located that it will 
pass through the centroid of all the test-vector termini, or it can be located 
so as to maximize the sum of the squares of the projections of the test vec- 
tors on the axis. The two latter methods would give nearly the same factor 
axis in Figure 1. As an example of the location of a first axis in the middle 
of the configuration, we have drawn the factor axis B; through tese 4. The 
resulting factor loadings are shown in the first column of Table 3,6. It will 
be seen that the factor loadings are all positive and of appreciable magni- 
tude. A common factor which has appreciable loadings on all the tests in a 
battery is called a general factor for that battery. Strictly speaking, a general 
factor might be one that has non-vanishing loadings in all tests, even 
though some are positive and some negative. But, as the term is ordinarily 
used, a general factor is one that has appreciable positive loadings in all 
tests of a battery. In the present problem the location of а general-factor | 
axis is not unique because it can be moved over a niaR ruben and 
still give appreciable positive loadings for all tests. B кке ү 
B, is placed somewhere in the middle of the configuration, the des or- 
thogonal factor axis By can be placed as shown or in the opposite \ ИКОНЫ, 
One of these directions is chosen as positive Cr B3) аза the I vs 8. 
tive (— B»). The same could have been done for the first axis—i i m лас 
S tors with negative projections. The factor loadings on By 
we It will be seen that some tests have positive load- 
ү е negative loadings pr iy d vi е factor 
aps ive saturations is called a bipolar factor. 
which has both positive iris ede a factor axis is placed in the middle 
of ie э ыы уг each of the subsequent factors i н ун j 
bipolar factors This theorem is clearly seen in the on ү ar * or , 
and it is tr | . a test configuration in any number of c ш sions or fac- 
» у ипе ory follows аза geometrical necessity from the location 
тсз = х * eii of the configurat ion of Hes Зра ме should 
18 ЧАН :ng to impose а physical interpretation on the bi- 
be cautious in attempting f bipolarity may have physical significance, 
о of pee LY eta consequence of the location of the 
ut it may also be n 4 


is i i test configuration. ү 
ror is y sene of the principles by which a frame of refer- 
e have const 


ctors c ay rati e shall now proceed t 
nce fa in the configur ation. W à £ 
rs can be located Ir 


been any test v 
are shown in Table 3, 
ings and that some tests hav 
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analyze two of these reference frames to illustrate the manner in which one 
attempts to find an interpretation for each reference factor. 

The frame А А» is represented by two unit vectors, A; and А». Each of 
these unit vectors may be thought of as a test vector for a perfect test of a 
factor. Such a test would have unit communality. The factor A, is of a 
nature such that it determines the intercorrelations of measures 1 and 3 
completely, since these two measures have zero loadings on the factor As. 
The factor A, is entirely absent in the variance of test 2, since the factor 
loading is zero. The factor А is involved in the variance of all the measures 
to an appreciable extent except No. 2, but it is represented in almost pure 
form in tests 1 and 3. 

If we know something about the measures that are involved, we turn to 
them to see if some hypothesis can be formulated for the factor that would 
seem plausible for all the tests that are high in the factor and which would 
also be absent in tests that are low in the factor. In psychological work one 
can sometimes make a plausible hypothesis by examining the tests, but 
more often it is necessary to use introspection for each test to ascertain the 
nature of the common factor. This is the method that has been used for 
identifying most of the primary abilities. 

In the present example it is assumed that we have no knowledge about 
the nature of the measurements. This is even more of a handicap than we 
ordinarily have in factorial studies, because experimental measurements are 
usually based on some knowledge of their implications, even if their under- 
lying order is not understood. If we arrange the nine different cylindrical 
forms in order, from the lowest to the highest in accordance with factor A1, 
we have the arrangement shown in Figure 2. This arrangement reveals that 
the factor is associated with the thickness of the cylinders, and we might 
call it the “thickness” factor. The diameter (1) and the cross-sectional 
area (3) are both determined by this factor, as is shown by the fact that 
these two measurements have no projection on the only other common fac- 
tor in this battery. The fact that the communalities are near unity reveals 
that there is hardly any residual or unique variance left in these two meas- 
urements after the factor A; has been determined. 

We proceed next in the same manner with the factor A » by arranging the 
nine different, cylindrical forms in order, as shown in the figure. These 
cylinders are evidently arranged in the order of their height, and we might 
therefore call this the “height” factor. This factor is involved in all the 
measures except 7 and 3. It should be noted that, when we name these fac- 
tors either by examining the nature of the measurements in some w 
examining the individual members of the experiment 


only a hypothesis concerning the nature of the f 


actor. Sometimes two or 
more hypotheses appear to be equally plausible for the interpretation of a 
factor. Then a new f. 


actorial experiment should be set up, with additional 


ay or by 
al population, we have 
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measurements so selected that they represent the two hypotheses with the 
least possible overlapping. The resulting factorial experiment might sustain 
one of the hypotheses, or it might show that neither of the hypotheses is 
correct. In the present case we have identified one factor that is associated 
with thickness. It has two scores or measurements completely determined 
by this factor, namely, 1 and 3 (diameter and base area). Another factor is 
associated with the height of the cylinder. We have found that all the 
measurements involve these two parameters or factors and that only one 
of the measurements (7) involves some additional factor. 

We turn next to the alternative reference frame represented by the unit 
reference vectors, By and Be. The factor By was placed so as to be collinear 


Аз [] | | [) Thickness factor 
^. [] [| L L | | | Tallness factor 


B, || C] LJ 8 || L| Size factor 
в miu 


short-thick short-thin and tall-thick 


1 Bipolar factor 
tall-thin 


FIGURE 2 


= з Ead iddle of the range of test vectors 
wW vet vector 4, which is in the mid | ge of А 
рава d 2 are positive and of appreciable magnitude, so that 
he factor loadings In Figure 2 we have the nine cylindrical forms ar- 
from the lowest to the highest. In- 


g to this factor, ‹ the h 1 
linders are arranged in order of size, irrespective 


this is a general factor. 
ranged in order accordin 


Spection rev hec r ol 
d ibus eal this characteristic а “general-size” factor. It can be 
Shape. We 


seen in the figure that the cylinders are in order раа аЬ 
Section area "This happens to be proportional tot i: ede К ^ шг taken 
for the factor B,, namely, the side area. wae : T ыыт ing 
vector as an index of size, such as the volume vec Я Р is 


i ration is not unique. | | | 
a in the агня Bs, according to which the nine cylindrical 
uc d oed in the last row of Figure 2. The plot of Figure 1 reveals 

s were arra 
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this as a bipolar factor, and we therefore look for some characteristic which 
can have either positive or negative values, or we look for a bipolar factor 
that can be thought of as a composite. For negative values of the factor В» 
we find the short-thick cylinders, while at the positive end of the distribu- 
tion we find the tall-thin cylinders. This might be called by a single name, 
such as a “slenderness” factor. Negative values for this factor would refer 
to short and thick cylinders. In the middle of the distribution, where the 
factor loadings are near zero, we should then expect to find those cylinders 
which are neither tall-thin nor short-thick. Here we find those that can be 
classified as short-thin and as tall-thick. The two descriptions, “short-thin” 
and “tall-thick,” would both have zero saturation on the slende 


rness factor, 
but tall-thick would have positive 


saturation on the size factor. For some 
purposes of classification or interpretation it might be meaningful to de- 
scribe these objects in terms of their general size, such as volume, and their 
degree of slenderness; but for most purposes this classification would prob- 
ably seem artificial. In most situations in which a configuration represents 
correlations that are positive or zero, we should probably find the simplest 
explanatory concepts to be represented by factorial axes that border or in- 
close the configuration rather than those in which a dominant factor is an 
average of all the measures of the battery by going through the middle of the 
configuration. As far as adequacy of description is concerned, any reference 
frame might be used which can gain acceptance as to the interpretation of 
the factors. All three of the frames that have been discussed here are equal- 
ly accurate in the description of the correlation coefficients. 

We have seen that the reference frame that is found in the factoring is 
ordinarily arbitrary. We have considered two alternative sets of orthogonal 
reference factors, both of which can be given some physical interpretation. 
The location of a meaningful reference frame is not unique, but there some- 
times exists a preferred reference frame that is more readily interpreted 
than the others. In the nature of the case one cannot prove that the frame 
chosen is in any sense the correct one. Nor can one demonstrate that some 
other frame would not also be meaningful. The best that one can do is to 
discover a set of reference factors that can be given plausible interpretation 
to aid in understanding the underlying order among the variables in the 
battery. Such findings can be followed by other experiments in the explora- 
tion of the factors that are revealed by the factorial analysis. It should be 
recognized that a factorial study is successful to the extent that meaningful 
factors can be identified by a proper choice of reference axes, 


CHAPTER VII 
THE SPHERICAL MODEL 


The vector model 

In previous chapters we have considered factor problems in two dimen- 
sions. The same concepts will now be extended to problems of three common 
factors. The correlation matrix can be represented by a set of vectors whose 
scalar products are the correlation coefficients. If the rank of the correla- 
tion matrix is 3, then there must be postulated three common factors. The 


JA 


© 


FIGURE 1 


vector model for such a correlation matrix would extend to three dimen- 
sions. In Figure 1 we have such a model, in which the test vectors are rep- 
resented by long pins or wires that are stuck into a central cork. Each of the 
pins represents a test vector. The correlation between any two tests is 
equal to is cos dis, where hı and Лз are the lengths of the pins and $i; is 
the angle between them. If such a model were given to us, we should be able 
to reconstruet the correlation matrix from which the model was built. No 
information is lost in going from the correlation matrix to the model or vice 


versa. They give the same information. 
125 
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In the vector model of the figure we see that the entire configuration af 
test vectors spans not more than a right angle. This means that the icum 
correlations are all positive. If two vectors in the model were at right angles, 
the corresponding correlation coefficient would be zero. The scalar product 
of a test vector with itself is h?, the angle $i: being zero, and cos фп, unity. 
Hence the square of the length of a test vector is equal to its communality 
hi. | 

In Figure 1 the test vectors are arranged more or less at random, except 
for the restriction that there are no obtuse angular relations. Such a con- 


Ficure 2 


figuration would be produced if we were to write a correlation matrix at 
random with positive or zero coefficients and subject to the restriction that 
it be of rank 3 to correspond with the three dimensions of the model. In 
Figure 2 we have a configuration of test vectors showing three clusters. The 
tests within a cluster would have high intercorrelations, while pairs of tests 
from different clusters would have low correlations. If the three clusters 
were at right angles, the tests from different clusters 
tions of zero or near zero. If such a configuration were found, it would be 
natural to suspect that the tests represented three different kinds of traits 


which are statistically independent. The clusters would be apparent by in- 
spection of the correlation matrix because the tests could 


would have correla- 


be so grouped 
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that the tests within a given group would be highly correlated, while tests 
from different groups would show low correlations. 

Figure 8 shows a different type of configuration, in that the test vectors 
are arranged in three planes. The configuration could be described as a 
three-sided cone with test vectors in each side of the cone and with apex at 
the center of the cork. If such a configuration of test vectors were found, it 
would be natural to suspect that three parameters or factors had determined 
the correlations and that the parameters could be represented in the simplest 
way at the corners of the configuration. Each test vector that lies in one of 


FIGURE 3 


the co-ordinate planes could then be deseribed as a linear combination of 


two primary or co-ordinate vectors. 
Wo primary or co-ordinate est vectors in the three sides 
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vectors are first extended to unit length. The termini of the vectors can 
then be represented as points on the surface of a unit sphere. Por the pur- 
pose of showing a test configuration graphically in three dimensions, one 
can use to advantage a blackboard sphere of the kind used in teaching spher- 
ical trigonometry. On the sphere of Figure 5 we have the same type of test 
configuration as in Figure 2, where the tests are arranged in three clusters. 
In Figure 6 we have the same type of configuration as in the vector model of 
Figure 3, where the test vectors lie in three planes. 

If we should find that the configuration of a test battery looked like Fig- 
ure 6, then the simplest description of the test vectors would be in terms of 


FIGURE 4 


the vectors A, B, and C at the corners of the configuration. All the tests on 
the side AB could be described in terms of the two vectors A and B; the 
test vectors on the side BC could be described in terms of the vectors B and 
C; and similarly for the test vectors on the side AC in terms of the vectors 
A and C, We should then make the hypothesis that the vectors A, B, and C 
at the corners of the configuration represent underlying properties or sim- 
plifying parameters in terms of which the correlations of the whole test bat- 
tery can be understood. A test vector near the middle of the side A B would 
then be described as a linear combination of A and B with equal weights 
for A and В. A test vector near A on the side AB would also be described 
as a linear combination of A and B, but the weight would be relatively larger 


FIGURE 5 


FIGURE 6 
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for A and smaller for B. In a configuration of this kind each test vector 
could be described in terms of less than three factors, even though the bat- 
tery as a whole required three factors. This is the principal characteristic 
of a simple structure, which is the combination of a test configuration and a 
set of co-ordinate axes in which each test vector lies in one or more of the 
co-ordinate planes, as in Figure 6. If there should be a test in the battery 
that involved all three of the factors A, B, and C, then it would lie inside 
the spherical triangle, andit would not contribute toward the location of 
the co-ordinate planes. 

In Figure 7 we have a model of test vectors in three dimensions in which 
the triangular configuration is incomplete. Here we have no tests which con- 


Ficure 7 


tain only the common factor A or the common factor C. The tests near B 
contain only one of the three common factors, namely, B. Furthermore, we 
see that all the tests in the battery have some saturation on the factor B 
because the tests on the side AB are linear combinations of A and B, while 
the tests on the side BC are linear combinations of B and C. Hence the 
factor B would here be a general common factor which is present in all the 
tests of the battery. In a configuration of this kind the location of the plane 
AC is indeterminate, and so are also the co-ordinate vectors A and C as 
far as the configuration is concerned. The only vector that is here deter- 
minate is the general common factor B.* 

* This common factor must not be confused with the kind of general common factor 


that is sometimes obtained for a factor matrix by locating the factor in the center of the 
configuration as a whole. Such an axis will necessarily be inside the spherical triangle, 
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Plotting the spherical configuration 

In order to study a configuration of test vectors in three dimensions, it is 
useful to extend them to unit length so that the configuration can be easily 
plotted on the surface of a sphere. When that has been done, we can see the 
whole three-dimensional configuration, as shown in the examples of Figures 
2, 6, 7, 10, and 11. The process of extending the test vectors to unit length 
Will be called normalizing the test vectors. j 

In Table 1 we have a given factor matrix of twenty measures or tests and 
three orthogonal factors, I, П, and III. The first step in normalizing the 


Table 1 
Normalizing of Test Vectors 
Facron MATRIX NORMALIZED Factor 
Р MATRIX 
Fr 

Test I I III ГА hj Wh; ї Il ш 
1|.6 .9950  .997497 1.002509 .138 
2 .0884 —.994183 1. —.660 
3 .9806 < 1. ‚505 
d .9951 < 1. —.444 
5 .0867 . 1. .511 
6 .9914  . 1. .153 
7 1.0094 1.004670 . —.268 
8 1.008 1.001500 .320 
9 Y. 1.001898 К —,908 
10 i. 1.007968 . —.838 —.351 
11 1. 1.001000 .888 —.147 .436 
12 1. 1.004788  . 871 .483 —.093 
13 "t 7991161 1.008918 | .673 —.731 “110 
14 1978673 1.021792 | .733 .251 —.682 
15 .961977 1.039526 | .65 .521 .543 
16 .084581 1.015660 | .951 .261 .168 
17 5 1.001403 | .967 —.239 —.083 
18 1.033292 | .646 —.744 ‚172 
19 1.038180 | .729 .116 —.675 
20 1.017288 | .675 545 -496 
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Each projection Aj, is the cosine of the angle between the normalized test 
vector J and the reference vector M. The relation between the given factor 
matrix F and the normalized factor matrix F, can be stated formally in the 
matrix equation 


(1) DF = Fag 


where D is a diagonal matrix with the stretching factor d;, or, by the cle- 
ments, 


(2) Ча» = Аз 


Before plotting the spherical configuration, we first draw on the sphere 
three mutually orthogonal great circles. If the sphere is to be used repeated- 


S 


5 


FicunE 8 


ly for factorial problems, it is best to paint or scratch the three great circles, 
so that they will not be erased for each problem. One of the octants is 
chosen as positive for all three co-ordinate axes I, IZ, III, as shown in 
Figure 10, and the three co-ordinate axes are labeled. 

In plotting the test vector termini on the surface of the sphere it is con- 
venient to use a strip of paper of length about twice the diameter of the 
sphere. On this strip are marked the surface distances corresponding to the 
cosines Aj. In Figure 8 we have represented the strip S, which has also 
been drawn in Figure 9. The point 1.00 on the scale is placed at the refer- 


ence axis J on the sphere. The point on the scale at axis IJ is marked zero. 
The surface distance from J to 77 is then 


(3) = = 0.7584 d = 1. 


> 
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lhe intermediate points on the scale are labeled directly in terms of the 
corresponding cosines, which are the entries in the normalized factor ma- 
trix Fa. For the projection Ад in Fa we have the corresponding angle 


(4) фл = costly 
and the corresponding surface distance 


5 tcosA; 
(5 [= I 
) pt 90 


For any given sphere the scale will be proportional to that of Figure 9. 'The 
measuring strip should be made twice as long as the scale indicated in Fig- 


70 60 50 40 .30 20 


FravnE 9 
ure 9, so that measurements can be made of both positive and negative 
cosines, 

In locating the terminus of 
Which has the required surface di 
The third measurement checks t 


a test vector on the sphere, we find a point 
stances from the three orthogonal axes. 
he location inferred from the first two 


Table 2 
Values of p for Given Values of Ajm = COS Фу 

a РА а ? 2 P $ ? e d 
“1.00 5 29 0 1 45 10 0 ST 
1-00 i Шо AS 205 55 0 4 15 — 80 
"BS 278 $5 60 150 .35 -7 10  .94 
97 6 30 ls 63 -30 81 .05 —.97 
"96 8 di cio 167 25 84 00 1.00 
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sphere shows that the points are arranged in the form of a triangle and that 
all but two of the test vectors lie in the sides of the spherical triangle. In- 
stead of using the arbitrary reference frame Г, ТЇ, III, which was obtained 
by the factoring method, we draw a new reference frame, as shown in Fig- 
ure 11. By using these new reference axes, we should be able to describe 
each of the test vectors that lie in the sides of the spherical triangle in terms 
of one or two reference axes instead of three, as required in the frame of 
Figure 10. The three great circles are so drawn as to pass through or near 
groups of vectors, as shown in Figure 11. The new reference vectors are the 
normals to the three great circles, and the primary factors are determined 
by the three intersections of the new great circles. 


Ficure 10 


With the model as shown in Figure 11 we are ready to put the factorial 
solution into numerical form. Consider the great circle drawn through or 
near the points 79, 2, 14, 9, 12, 6, 20, 8, 15. By means of the measuring 
strip locate a point on the surface of the sphere which is orthogonal to this 
circle. It represents one of the unit reference vectors, which may be denoted 
Az. In the same manner locate the other two reference vectors orthogonal 


to the other two planes. One of the reference vectors is shown in Figure 11, 
namely, A.. 


By means of the measuring strip, measure the three direction cosines for 
each of the reference vectors and record them in columns of the transforma- 
tion matrix A, as shown in Table 3. In determining the direction cosines of 
areference vector, measure its surface distance from each of 


the three orthog- 
onal axes I, II, ITI, on the sphere. In the first sec 


tion of the table we have 
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the given factor matrix F, and in the last section we have the desired factor 
matrix V, which describes the twenty test vectors in terms of the three new 
reference vectors Ap, Where the subscript p refers to the new axes. When the 
measurements have been taken from the sphere, it is advisable to adjust 
them in the second or third decimal, so that the sum of their squares is 
equal to unity. This step insures that each of the reference vectors is of 
unit length. 

When the transformation matrix A has been written as in Table 3, we 
can carry out the matrix multiplication 


(6) FA-V. 


FravnE 11 


atrix show the projection of each test vector on each 
of the three new reference vectors. The point 20; for example, is on or near 
two of the new co-ordinate planes, as shown in Figure 11, and hence it 
should have two factor loadings that are zero or near zero, as shown in 


Table 3. 
The new reference vect 


The entries in this m 


ors are not orthogonal. The cosines of their angu- 
lar separations can be determined by the measuring strip and recorded E 
the matrix C of Table 3. The numerical values of these cosines can also be 
determined without direct measurement by the equation 


(7) 
or, in terms of the elements, 
(8) 


AA RC, 


„Аа = Cpa» 
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where the subscripts p and q refer to the new axes. The transformation 
matrix A is not an orthogonal matrix, because the three new reference vec- 
tors are not orthogonal. The columns are normalized to correspond to the 
fact that the reference vectors are unit vectors. The cross products of pairs 
of columns of the transformation matrix A give the cosines of their angular 
separations, as in equations (7) and (8). In an orthogonal matrix these co- 


Table 3 


The Transformation Matriz A 


Given Factor Matrix Р Transformation A Rotated Factor Matrix V 
Test| I II III | Az Ay As Test| Az Ay 

1 | .659 —.736 .138 I -483 .466 .479 1 .969 .003 —. 
2 | .725  .180 —.656 IL|—.834  .254 .560 2 025  .939 

3 | .665 .537  .500 Ш .267 —.847 .675 3 007 ‚023 

4 | .869 —.209 —.443 1 476 .727 

5 | .834 .182 .508 E aa 5 387 .005 

6 | .8386 .519 .152 6 012  .393 

7 | .856 —.452 —.269 7 719 .512 — 
8 | .848 —.426  .320 8 850  .016 

9 | .861 .416 —.299 9|-.011 .760 
10 | .880 —.341 —.354 10 615 .623 — 
11 | .889 —.147  .436 11 668 .008 
12 | .875  .485 —.093 12 |—.007 610 
13 | .667 —.725  .109 13 | .956 .034 — 
14| .717  .246 — 619 14 |—.024 .921 

15 | .634 .501 |5292 15 028 —.019 
16 | .936  .257  .165 16 282 362 

17 | .966 —.239 —.083 17 644 460 
18 | .625 —.720 166 18 947 —.032 
19|.702 .112 —.650 19 072 906 — 
20 | .664 .536 .488 20 | .004 032 
a = гк 


Primary Vectors T 


TA=D 
ee es Е Сре E 
Т. | „661 —.736 .144 Т. 0 
T,| 1713 .1099—6й ele d л 
T: | .654 1.545 1524 T 0 К) .972 


Correlation of Primaries Cosines of Reference Vectors 


| Tz Ty Tz | Az Ay Az 


| 1.000 -229 .105 Ал, | 1.000 — .213 — .055 
| 1229 1:000 ‘294 — .213 1.000 — :906 


9 А; 
105  .294 1.000 А, 055 — .906 1:000 
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sines аге zero because such a transformation represents the rotation of a 
rigid orthogonal reference frame. 

The primary vectors which represent the three underlying factors or 
parameters in pure form are unit vectors at the corners of the configuration. 
A unit vector at the corner near the test vectors 3, 15, and 20 represents one 
of these primary factors. It is defined by the intersection of two of the co- 
ordinate planes. Let the primary test vectors be denoted Ta, Ty, and T.. 
The direction cosines of each of these unit vectors may be determined by 
the measuring strip and recorded in the rows of the factor matrix, as shown 
in Table 3. Here we are treating the three primary vectors as if they were 
additions to the given factor matrix. If we had three perfect tests of these 
primary factors with no unique variance, they would appear in the factor 


matrix, as shown in the rows T+, Ty, and T.. 
If we add the three primary vectors to the given factor matrix and carry 
out the transformation, 


(9) FA = У, 

for the last three rows of the factor matrix, we have the corresponding 
equation, 

(10) TA=D, 


where the product must be a diagonal matrix, as shown in Table 3. By defi- 
nition each primary vector lies in two of the co-ordinate planes, and hence 
it must have zero projections on two of the reference axes. The result is a 
diagonal matrix. If the reference vectors were orthogonal, they would coin- 
cide with the primary vectors; but, when the reference vectors are oblique, 
as in the present case, the product is a diagonal matrix, in which the diago- 
nal entries deviate from unity. The diagonal entries are, in fact, the cosines 
of the angles between the primary vectors T, and the corresponding refer- 
ence vectors Ау, as ean be verified on the spherical model. 

The matrix T, which shows the direction cosines of the primary vectors, 
can be determined from the transformation matrix A without direct meas- 
urement on the sphere. By (10) we have 


(11) T = рл. 
We find the inverse of the transformation matrix A and normalize the rows 
of that matrix. Normalizing the rows of A7 is represented in equation 
(11) by the premultiplication with the diagonal matrix D. The result is the 
matrix T, which shows the direction cosines of the primary vectors. 

The correlations between the primary factors can be determined by di- 
rect measurement on the surface of the sphere. The same correlations can 
be determined from the transformation matrix A without direct measure- 
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ment. Just as the correlations between the tests are given by the funda- 
mental factor theorem, 


(12) FF’ = Ва = В, 


so for the last three rows of the factor matrix we also have the correspond- 
ing equation, 


(13) DT В. 


Substituting (11) in (13), we obtain 


(14) Ry = ØA IDA 
or 
(15) Rp = Р(А'А)-1р. 


By this equation the correlations between the primary factors can be deter- 
mined. When the product (A’A)~! has been computed, the diagonal matrix 
D is so written that the diagonal entries of the correlation matrix Rpa are 
unity, since the scalar product of a unit vector and itself is, of course, unity. 
The diagonal matrix in (15) is the same as the diagonal matrix shown in 
in the lower section of the oblique factor matrix V. Each entry in D is the 
cosine of the angle between a primary vector T, and the corresponding 
reference vector Ap. 

For some purposes it is useful to designate explicitly the general ele- 
ments of the several matrices of Table 3. In the given orthogonal factor 
matrix we have the elements аы, and in the transformation matrix we have 
Amp, Where the subscript p refers to primary factors. The product is 


(16) ау Amp = Vjp у 


where the elements v;, of V show the projections of the test vectors j on the 
reference vectors Ap. The elements of the matrix T are denoted tpm. The cor- 
relation matrix for the primary factors is denoted R pa, to distinguish it from 
the correlation matrix for the tests, which is written Rir, or simply R with- 
out subscript. Let the elements of (A'A)7! be denoted bpa. Then the diagonal 
elements are b,,. The diagonal elements of D are, then, d,, or 


ii 
d, = Vin ғ 
The present example has been discussed in detail for the three-dimension 
al case to show the relation between the computational work and the geo- 
metrical interpretation. In the interpretation of primary factors we inspect 
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each column of the oblique factor matrix V to discover, if possible, the na- 
ture of the trait which is present in the tests with significant factor loadings 
and absent in the tests with negligible loadings on the factor. There is no 
guarantee beforehand that a simple configuration like Figure 11 can be 
found in any given problem, and there is no guarantee that a clarifying de- 
scription of the factors can be obtained. That is determined by the intui- 


lions of the investigator in setting up his factorial experiments so that 


meaningful and significant results ean be obtained. 


Correlated factors 

In the previous discussion, examples of both orthogonal and oblique ref- 
erence vectors have been given. A set of orthogonal reference vectors is a 
geometrical representation of factors which are uncorrelated, and corre- 
lated factors are shown by an oblique reference frame. Among statisticians 
and psychologists there is a rather general belief that if human traits are to 
be accounted for by any kind of factors, then these factors must be uncorre- 
lated. This belief has its origin in the statistical and mathematical conven- 
ience of uncorrelated factors and also in our ignorance of the nature of the 
underlying structure of mental traits. The reason for using uncorrelated 
understood, but it cannot be justified. Height and 
asures of body size, even though they are corre- 
lated. If we should use, instead, two linear combinations of height and 
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sary it would be to insist on zero correlation between parameters when the 
relations are well understood. Let us suppose that nothing is known about 
the nature of rectangles except that we can take measurements on them. 
We might measure five hundred rectangles and discover that there is a corre- 
lation of +.70 between the lengths and the widths in our collection of rec- 
tangles. Such a finding would be determined by the fact that big rectangles 
tend to have long sides and to be wider than small rectangles. We might 
have the conviction that the principal properties of rectangles could be de- 
termined from their lengths and widths, but we should have to mect the 
restriction that the two parameters must be uncorrelated in the particular 
set of rectangles that we collected for measurement. That could be done if, 
for instance, we took the length as one measurement and, say, (HW —1L) as 
the other measurement, where L and W are the length and width, respective- 
ly. The linear combination would depend on the particular set of rectangles 
in the experiment. Two measures could easily be found which were uncor- 
related in our particular collection of rectangles, but the result would not 
be very useful in thinking about rectangles. As another example, in taking 
physical measurements of men, it is customary to record height and weight. 
If we were forbidden to use these two measures because they are correlated, 
the absurdity of such a restriction would be self-evident. On the other hand, 
it should be recognized that if we deal with a group of measures that are 
highly correlated, we are likely to be confining ourselves to one domain, and 
it is then useful to look for other significant measures that cover additional 
features of the things that are being measured. In general, measures that 
cover different aspects of an individual are likely to show lower correlation 
than those which represent the same domain; but it does not follow that 
our fundamental concepts should be in any sense restricted to those which 
show zero correlation in a random group of people. In developing the fac- 
torial methods we have insisted that the methods must not impose ortho- 
gonality on the fundamental parameters that are chosen for factorial de- 
scription, even though the equations are thereby simplified in that the cross 
products vanish. In the methods that we have developed, each author is 
still free to impose orthogonality on the factors if he so chooses or if the 
nature of his problem is such that this restriction is indicated, 


The box problem 
The box problem is an example involving three parameters or factors 
that are represented in the spherical model of Figure 11.* A physical ex- 


* The numerical problem that is used in this chapter is the so-called 
which was designed to correct some current misconceptions about factor analysis. Sec- 
tions from the original publication of the box problem will be used here to illustrate by a 
physical example the interpretation of primary factors as simplifying parameters that 
are not necessarily unique (see “Current Issues in Factor Analysis," Ps 
tin, XXXVII, No. 4 [April, 1940], 189-236). 


"box problem,” 


ychological Bulle- 


THE SPHERICAL MODEL 141 


ample of two parameters or factors—the cylinder problem—was used in 
chapter vi. s 

A random collection of boxes constitutes the population in this example. 
Let us imagine a collection of boxes and a set of twenty measurements for 
ach box. Let us assume, as is really the case in most psychological problems, 
that we have no idea what these measurements represent and that we believe 
some underlying order may exist in the twenty measurements for each box. 
In this fictitious example the boxes constitute the individuals in a statistical 
population, and the measurements correspond to twenty test scores for each 
individual. In Table 4 we have the three dimensions, т, y, z, for each box. 


T'able 4 
Dimensions 

Boxes په‎ 
x У 2 

1 3 2 1 
2 б 2 R 
3 3 3 1 
4 3 3 2 
5 3 3 3 
6 4 2 1 
7 4 2 2 
8 4 3 1 
9 4 3 2 
10 4 3 3 
11 4 4 1 
12 4 4 2 
13 4 4 3 
14 5 2 1 
15 5 2 2 
16 5 3 2 
17 5 3 3 
18 5 4 1 
19 5 4 2 
20 5 4 3 


The box population is here defined in terms of twenty different rectangular 
box shapes, and it is assumed that these twenty shapes occur with equal fre- 
quency in the population of boxes. For example, if the population consists 
of two hundred boxes, there are ten of each kind. Any other arrangement 
could have been taken. In Table 5 we have a list of the twenty measure- 
ments (tests) that were taken for each box. These measurements consist of 
various non-linear functions of the three dimensions, т, y, 2; but in the factor 
analysis the number of basie factors and their combination in the twenty 
measurements are assumed to be entirely unknown. 

It will be seen in the formulae of Table 5 that each of the tests 1, 2, and 3 
represents the square of an edge of the box; 4, 5, and 6 represent the area of 
and 9 represent the diagonal of a side; 10, 11, and 12 represent 


a side; 7, 8, 
a side; and the other tests are similar arbitrary functions 


the perimeter of 
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of the three box measurements. In dealing with the twenty о 
for each box, we treat them merely as if they were twenty test ARR ge 
each box. We correlate them and factor them as if we knew nothing a iod 
the manner in which the scores became correlated. Our object is to discov S 
by factor analysis whether or not the twenty measurements for each box 
reveal some underlying physical order. Actually, we have set up the meas- 
urements so that three factors are involved—namely, the three dimensions 
of a box—but we deal with the problem factorially as if we did not even 
know how many factors were involved in the twenty test scores for each 


Table 5 


Structure 
Tests سے‎ Test Formulae 


x » z 
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log y 
log z 


= 
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box. In order to simplify the problem, е 


rrors of measurement and sampling 
errors are here omitted. The corre] 


ations are therefore unaffected by the 
number of multiples of the twenty different box shapes that constitute the 
statistical box population. The correlations were determined from the box 
measurements listed in Table 4 and the test formulae of Table 5. In Table 3 
we have the centroid matrix for three factors. The mean of the absolute 
residuals (disregarding sign) after three factors had been extracted was 
-008, which does not warrant the extraction of more factors. At this point 
we would know that only three factors were involved in the twenty fictitious 
box scores. 


In the interpretation of the rotated factor m 
the tests which have the high saturation 
tests 1, 13, and 18; and we conclude th: 


atrix V of Table 3 we look for 
s in the first column. These are 


ıt the first primary factor is very 
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heavily represented in these tests. In this fictitious problem we can actually 
turn back to Table 5 to find the formulae for these tests in terms of the pri- 
mary factors. These are, of course, not known in psychological problems. 
We find that tests 1, 13, and 18 are the only three tests which depend ex- 
clusively on the z-dimension of the boxes. 

In the third column of the factor matrix V of Table 3 we find three tests 
with high saturations, namely, 3, 15, 20. These three tests are therefore 
judged to have most of their variance on one of the underlying factors. In 
Figure 11 we find these three tests at one corner of the configuration, and we 
conclude, therefore, that they involve only one of the three common factors, 
whatever they may be. Turning back to Table 5, as we can do in this ficti- 
tious example, we see that the three test scores are the only ones that are 
functions of the parameter z. 

Consider the bottom row of tests in Figure 11. These tests are 19, 2, 14, 
9, 12, 6, 20, 3, 15. Since these test vectors lie in one of the co-ordinate 
planes, we infer that there is one factor that is absent in all these tests and 
that the factor is conspieuous in 1, 13, and 18, which lie at the opposite 
corner of the configuration. Turning to Table 5, we see that this factor is z. 

The same kind of comparison can be made for each of the three primary 
factors x, y, and z. Those tests which depend on only one of the primary 
factors are in the corresponding corner of the diagram, and they are repre- 
sented in the factor matrix V with large factor loadings in the proper col- 
umn. Those tests which are functions of two primary factors are represented 
as points on the sides of the triangle, and the points are consistent through- 
out as to the pairs of primary factors involved. Finally, the two tests 16 and 
17, which have formulae containing all three of the primary factors, are in- 
side the triangle. They are the volume and the principal diagonal, respec- 
tively. It is interesting that the linear approximations which are assumed 
in factor analysis reveal a simple configuration, even though the twenty 
measurements were nonlinear functions of the primaries. The discrepancies 
are represented by the small entries in the factor matrix V, which would be 
zero if the factor methods did not depend on the assumption of linearity as 
a first approximation. 

Next we note that the three primary factors in this population of boxes 
are correlated. The correlations among the primary factors as determined 
by the factor analysis are shown in Table 3. The actual correlations between 
the primaries in the original data are shown in Table 6, and the agreement is 
self-evident. In setting up this example we assumed that a random collec- 
tion of boxes would show correlation between the three dimensions. A box 
that is tall is likely to be thick and wide, probably to a greater extent than 
the correlations which we assumed in the fictitious problem. 

It would be possible to set up an example with twenty measurements 
so that every measurement would involve all three of the box dimensions. 
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A factor analysis of such a system of measurements would not reveal a sim- 
ple structure, so that no clear bounding planes could be determined. In 
psychological experiments we must recognize this possibility. Some factor 
experiments will fail to reveal primary factors. This will necessarily happen 
if each variable involves as many factors as there are factors in the whole 
set of variables. There is no guarantee beforehand that this will not happen. 
On the other hand, it is extremely unlikely that a simple structure will ap- 
pear with the same identifiable primaries in several independent factor anal- 
yses of different test batteries that are given to different populations, unless 
the structure is valid. 

The interpretation of the primary factors in the factor matrix V would 
consist in studying the several test variables that show high saturation on 


Table 6 


Correlation Matrix 


| X Y 2 
Х 1.00 

.25 1.00 
Z | 10 25 1.00 


each primary factor. One would try to find what it is that is common to 
these variables or measures. One might discover that, in general, tallness is 
à common characteristic of one primary and that the best measurements 
available for tallness are those which show highest saturation on the factor. 
We might name it a “tallness”” factor. That would merely imply that the 
twenty measurements are functions of three linearly independent parameters 
and that one of them is tent atively named “t 


ures high on that factor would st and higher from the floor than boxes that 
are low in that factor. Now this identification of the linearly 
parameters in the twenty box measureme 
ness factor has some sort of locus in each box or that the factor is ¢ rate 
organ or gadget in each box or that it is some sort of isolated, abstract, sta- 
tistical box-element that has nothing in common with the rest of the box. 
It is an independent parameter that is involved in the box measurements. 
Exactly the same procedure is followed in dealing with psychological data. 
The identification of a primary factor in psychological test experiments is 
merely a challenge for us to find out what sort of process or attribute it is 
in terms of known concepts or else to invent new concepts that fit the ex- 
perimentally determined primary categories. 

The attempts to interpret the primary factors h 
because they are not sufficiently objective, T 
tinct problems that are involved here. 


allness.”” A box which meas- 


independent 
nts would not imply that the tall- 


ave often been criticized 
^et us be clear about two dis- 
It is one problem to isolate a primary 
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factor and to determine by repeated experiments that it has some functional 
uniqueness. That is factor analysis proper. It is another problem to find 
the psychological or physiological meaning of a functional uniqueness when 
it has been determined. That is a matter of interpretation, and consequent- 
ly it is necessarily subject to debate, with conflicting interpretations. We 
must remember that the interpretation of every scientific experiment is sub- 
jective. There is no kind of scientifie experiment in which the interpretation 
rolls out “objectively.” If there is such a factor as auditory imagery, for ex- 
ample, just how should the auditory character of the factor produce itself 
objectively in the statistical analysis and without the subjective interpreta- 
tion of the investigator? Or just how could tallness in the box population ap- 
pear objectively in the data? Factor analysis is no exception to other kinds 
of scientific experimentation. It is a fortunate circumstance, however, that 
different interpretations of a primary factor can usually be resolved as ques- 
tions of fact. New factorial experiments ean be made to determine which 
interpretation is the more plausible. 

In naming the primary factors it is a better policy to name the factors in 
terms of well-known concepts, such as Number, Space, Verbal, and Memory 
factors than to name them in some noncommittal way, such as ai, 2%, and 
soon. If we name a factor “Number,” it will provoke experimentation with 
number tests and with nonnumerical tests, and the experiments are likely 
to be made in terms of psychological hypotheses that can be sustained or 
disproved experimentally. In this way we shall advance faster than if the 
primary factors are left as interesting statistical curiosities. They should be 
recognized as psychologically challenging experimental effects. We cannot 
expect to be correct in all interpretations of primary factors, and we shall 
probably have occasion to revise interpretations repeatedly as more experi- 
mental information becomes available. Again, this policy is consistent with 
the desire to make factor analysis a useful psychological tool instead of a 
mere statistical routine for condensing test scores and correlations. 

Those who criticize factor analysis as faculty psychology talk in the next 
breath about verbal and nonverbal intelligence, special aptitudes for music 
and for art, mechanical aptitudes, and disabilities in reading and arith- 
metic, without realizing that they are implying factor analysis and the inter- 
pretation of factors. Conversationally, we ean talk about musical ability in 
the singular; but, as psychologists, we believe that different abilities are 
probably involved in a good voice, absolute piteh, originality in harmony, 
counterpoint, orchestration, melodie memory, ease in memorizing at the 
piano, ability in musical interpretation, and so on. How many important 
functional unities might there be in this domain? They are surely not all 
completely independent. ‘They are correlated, and they probably represent 
a smaller number of abilities than the infinite number of musical tasks that 
student. To find these functional unities is the 


could be given to a musi 
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problem of factor analysis. If that is faculty psychology, then so is most in- 
vestigation of individual differences and most of the work on special apti- 
tudes and defects. And so also are many of the chapter headings in the 
textbooks of psychology which the same critics are teaching. 


Correction for uniqueness 


The correlation between two experimentally obtained measures is some- 
times called a “fallible” coefficient in the sense that the correlation is some- 
what depressed by the experimental errors in the measures. If the two 
measures are the best available for two traits т and y and if it is desired to 
estimate the true correlation between the traits themselves as it would be 
expressed by perfect tests of the two traits, then the obtained or fallible co- 
efficient is corrected for attenuation. The correction for attenuation is an 
estimate of what the correlation between two tests would be if the error 
variances were eliminated. The estimate is made in terms of experimentally 
determined reliability coefficients for both tests. If a factor matrix is avail- 
able showing the loadings on common factors, specific factors, and error 
factors, then the correction for attenuation can be obtained by eliminat- 
ing the error factor loadings and then normalizing the remaining common 
factors and specific factors for each test, so that the sum of their squares 
is equal to unity. The correlation between two tests corrected for attenua- 
tion would then be the cross product of their respective rows in the normal- 
ized factor matrix for common and specific factors. The same numerical re- 
sult is obtained by the usual formula for this correction, namely, 


(17) i Cu 
LJ Nas уу. ' 


where 74, is the augmented coefficient, corrected for attenuation, and r, 
and ry, are the reliability coefficients. 

The correction for uniqueness is analogous to the correction for attenua- 
tion. Instead of eliminating the error variance, we can imagine the test so 
modified as to eliminate both error variance and specific variance. The re- 
sult would be a measure containing only the common factors normalized 
so that the sum of the squares of their loadings is unity. That is what Ta- 


ble 1 represents. The correlation between two tests as determined by the 
normalized common factor matrix is 


(18) Ru ccr Try 


м huh 


in which communalities are substituted for reliabilities. The correlations 
corrected for uniqueness are the scalar products of pairs of unit test vectors 
аз represented on the spherical model. These corrections are useful in some 
theoretical problems, and they are represented in the illustrative problem 


NL E 
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of this chapter by the cosines of the angles between the normalized test vec- 
tors. In the present case the principal interest is in studying the configura- 
tion of test vectors and not in estimating what the correlations would be if 
the tests were in any way altered. In studying the configuration of test 
vectors by the methods described here, it is best not to include in the con- 
figuration any tests whose communalities are small. Such test vectors are 
short, and, when they are much extended, their projected location on the 
surface of the sphere is unstable. If they are used in plotting the configura- 
tion, they should be specially marked so that less weight is given to them 
than to the test vectors which are not very much extended in normalizing. 
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bination of two reference vectors lies in the plane that is spanned by the two 
reference vectors. The test vector can then be defined by a vector equation, 
such as 


(19) Wj Ё А a = J, 


in which wj; and wj; are the weights assigned to the test vector J for the 
two reference vectors X; and Ху. If the two weights are equal, then the 
test vector J lies midway between the two reference vectors in the plane 
I-II of Figure 12. If the weight wj, is the larger, then the test vector J lies 
close to the first reference vector. If we write the equation of a test vector J 
for the three dimensions of Figure 12, we have 


(20) юл Ху + Хь + 03: = J, 


in which the weight wj; is zero when the test veetor J lies in the plane ITT. 

The signs of the three weights determine the region in which the test 
vector J is located. If all three of the weights are positive, the test vector J 18 
located inside the positive octant, as indicated by the signs + + + in the 
positive octant of Figure 12. If the third weight is zero and the other two 
are positive, we have the weights -+ + 0, and the test vector then lies in the 
are I-IT of Figure 12. If the weights are 0 + 0, the vector J is collinear with 
the second reference vector, as shown in the figure. Similarly, if the weights 
are + 00, the vector J is collinear with the first reference vector. If the 
first weight is negative, we have thew eights — + 0, and the vector J then 
lies in the plane spanned by the first and second reference vectors, as shown 
in the figure. Similar relations hold for the weight combination + — 0 as 
shown. 


There are twenty 


-six sign combinations of the weights in equation (2) 
which define the reg; 


ion in which the test vector can be located in three di- 
mensions, the combination 000 being eliminated because it represents a 
null vector. If one of the signs is zero, the test vector lies in one of the со- 
ordinate planes. If two of the signs are zero, the test vector is at the inter- 
section of two of the co-ordinate planes. The same reasoning can be extend- 
ed to n dimensions. The fact that the weights of the equation are all posi- 
tive does not imply that the configuration сап be inclosed within 
spherical triangle unless the reference vectors 
or acute angles. If the saturations of the 
positive and if the factors are not negativ. 
tion ean be inclosed in a right spherical tr 
octant. This is a special case of considerable psychological interest because 
of the fact that mental abilities do not seem to show any negative correla- 


tions and psychological tests are positively correlated, even when they rep- 
resent the most diverse t 


a right 
are separated by right angles 
tests in the several factors are all 
ely correlated, then the configura- 
angle which becomes the positive 


asks, 


CHAPTER VIII 
NTROID METHOD OF FACTORING 


THE CI 

In factoring a correlation matrix the purpose is usually to account for 
the correlations with fewer factors than there are tests, so that r <n, where r 
is the number of factors and n is the number of tests. With fallible data 
there will always be residual correlation coefficients after the extraction of r 
factors unless r=n, in which case the residuals сап be made to vanish ex- 
actly. The reason is that the rank of a correlation matrix for fallible data is 
always equal to its order. The factoring should be done so as to make the 
residuals as small as possible after each factor has been determined. The 
least-squares solution for this problem minimizes the residuals and maxi- 
mizes the factor loadings on each successive factor. The resulting factorial 
axes are the principal axes. These would be determined routinely in factor 
analysis, except for the computational labor that is involved for large test 
batteries of twenty or thirty or more tests. The centroid method, like the 
grouping method, is a computational compromise. The main centroid axis 
for any given table of correlation coefficients, or residual coefficients, can be 
regarded as a first approximation to the major principal axis. The main cen- 
troid axis for any given correlation matrix, or residual matrix, is obtained by 
simple summational procedures after appropriate reflections of the test vec- 
tors with corresponding sign changes, and the results are often fairly close 
to the statistically ideal least-squares axes. In large problems it is sometimes 
desirable to determine an additional factor by the centroid method beyond 
what is required by the principal axes to obtain the same average magnitude 
of residuals. The subsequent rotation of axes from either method of factor- 
ing usually shows one or more residual factors that do not contain enough 
variance for dependable interpretation. 

Several theorems will be derived which involve the centroid of the test 
vector termini. These will be followed by numerical and geometrical exam- 
ples. The theoretical examples in this chapter are exact, in that all correla- 
tional elements are assumed to be known and the rank r is less than n, so 
that the rth-faetor residuals vanish identically. An example will also be 
given showing computational methods that are suitable for experimental 


data. 
Centroid theorems 
Consider, first, the fundamental equation of multiple-factor analysis, 
namely, 
(1) Tik = адак + аза + +++ + аа, 
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in which the correlation between two tests, j and k, is expressed in terms of 

the factor loadings of these tests in r orthogonal factors. Summing each 
р gs 5 

column of the correlation matrix, we have 


— < NA 
(2) > Tjik = am У aj + ais У aj sss bay 2 а}. 
j 7 7 j 


Summing for all columns, we have 


— X E EY 4 4 SEA ДСУ. NAE 
(3) A Ans SMa ууаа + Yny aja + + ак , йе 
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But 
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so that 
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Let the sum of all the correlation coeffie 
noted тү, then 


(6) >, Уту ый Г 
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so that 
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2 


ients (or residual coefficients) be de- 


We then have the interesting theorem that the sun 
correlation matriz is equal to the sum of the squares 
corresponding orthogonal factor matriz. 

For the purpose of illustrating the theorems of this ch 
tious example was written, as shown in T. 
factors. The intercorrel 
lations is +.29, A 


1 of all the coefficients in a 
of the column sums in any 


apter, a small ficti- 
able 1, for five variables and two 
ations are shown in Table 2. The sum of the corre- 
pplying the theorem to the sums of the factor matrix, we 
have (—.2)*2- (4-.5)?— +.29, which is also the sum of the correlations. In 
Figure 1 we have the co rresponding configuration of five test vectors, which 
are drawn without reference frame. 


THE CENTROID METHOD OF FACTORING 151 


Table 1 
Fictitious Factor Table 2 

Matrix Correlation Matrix 

I п +1 +2 +3 +4 +5 

1|—.9 —.1 +1 .82 .60 — .33 — .64 — .58 
2| -.6 —.6 +2 .60 72 — .06 — .48 — .60 
3 4 =8 +3 | = .33 — .06 .25 25 12 
1 7 1 44 | = :64 — 48 25 50 46 
5 6 4 45 | —.58 — .60 12 46 52 
x|-.2 5 = .18 18 .23 00 — .08 m=.29 


FIGURE 1 


Tho centroid of a set of points is the center of gravity of equal weights at 
the points. The centroid of the test vector termini has the co-ordinates 
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If the axes are so placed that the centroid lies in the first axis of reference, 
then the centroid has zero projections on all the remaining (r— 1) co-ordi- 
nate axes. Hence, 


(8) San = Уа = Dae 0, 


A J J 


so that the r co-ordinates of the centroid are 


1 
„ 259 0,0. ..,0 
j 
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Substituting (8) in (7), we have 


(9) т, = PE 


j 


where the first axis contains the centroid. We then h 
the sum of the coefficients in the correlation matrix is eq 
sum. of the first centroid factor loadings. 

The first co-ordinate of the 
distance from the orig 
Hence the distance 


ave the theorem that 
ual to the square of the 


centroid of the test vector termini is also its 
in, since the remaining (r—1) co-ordinates vanish. 
of the centroid from the origin is 


(10) iiS a. 
N ست‎ 
2 
By (9) we have 
(11) d= - vr, 
1 


in which the distance of the cer 


troid from the origin is expressed in terms of the 
coefficients in the correlation n 


atria, 
In order to determine the loadings ад on the centroid factor, substitute 
(8) in (2). Then we have 
= 
(12) Di = а > ал A 
J 


and from (9) we get 


(13) bar = а Vri. 
j 
For convenience, let the column sums of the correlation matrix be denoted 
(14) 2, lik = ть. и 
Then 
(15) Te = an Vr, 
so that 
(16) аа = T 
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l'or computing, determine a multiplier » = ——. Then 
Un 


(17) ам = тт, 


by which the first-factor loadings are proportional to the corresponding 
column sums of the correlation matrix. By (17) the first-factor loadings are 
numerically determined, 

By equation (9) we have a useful check on the computations, namely, 


у= 
(9а) > аа = Уг. 


j 


Reflection of test vectors 

The correlation matrix of Table 2 represents a configuration which is 
shown in Figure 1. The fact that no reference frame has been drawn in this 
figure corresponds to the fact that no reference frame is implied in the cor- 
lation matrix. A first axis in the configuration of five vectors evidently 
should be in the general direction of the vectors 7, 2, 4, 5, since an axis in 
this direction through the configuration would make the projections the 
largest possible. It is of secondary consequence whether the positive sense 
of this first reference axis is in the direction of the test vectors /, 2, or in the 
direction of 4, д. 

Since the configuration is not immediately given by the correlation ma- 
trix and since it cannot be physically constructed or seen when the corre- 
lations involve more than three dimensions, it is a practical problem to de- 
termine the general direction of the longest dimension of the configuration 
by inspection of the correlation coefficients. When that can be determined 
by inspection of the correlation matrix, several of the test vectors can be re- 
flected so as to move the centroid as far away from the origin as possible. 
Then the first centroid axis can be located so as to account for a large part 
of the correlation coefficients, leaving their residuals as small as possible. 

Several criteria can be used to ascertain which test vectors lie in or near 
the longest dimension of the configuration. These criteria are not always 
consistent, since a configuration has extension in several directions. One 
criterion is to determine the absolute sum of the correlation coefficients, or 
residual coefficients, in each column. That test vector which has the highest 
absolute sum is likely to be in or near the longest dimension of the test con- 
figuration. Another eriterion is to find that column which has the largest 
absolute sum, ignoring the diagonals. One simple method is to select for re- ^ 
flection that test vector which has the largest number of negative coeffi- 
cients. This process can be continued until each column has a majority of 
positive coefficients. Another method is to select for reflection that test 
vector which has the highest communality. Inspection of the correlation 
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matrix of Table 2 indicates that test vectors 1 and 2 should be reflected. 
When these columns and rows have been reversed in sign, the resulting cor- 
relation matrix is shown in Table 3, and the configuration, after reflection, is 


shown in Figure 2. The centroid of the reflected configuration is shown at 
point ¢ in Figure 2. 


Ficurr 2 

The computation of the first-factor lo 

columns to find ть for each column. F 

m. Apply the multiplier m to find 

These loadings correspond to the vari 

The same factor loadings are recorded in the 
variables with positive sign. 


adings is shown in Table 3. Sum the 
ind the total sum 7, and the multiplier 
the first-factor loadings aj: as shown. 
ables with signs, as shown in Table 3. 
factor matrix of T'able 4 for all 


Table 3 Table 4 
Correlations after Sign Changes Factor Matrix 


-1 -2 +3 +4 +5 I п 

ail! ‚82 -60 .33 -64 -58 1 | —.893 —.149 
-2 .60 .72 .06 .48 .60 2 | —.740 414 
+3 -33 .06 .25 .25 .12 3 .304 .397 
+4 ‚64 .48 .25 .90 .46 4 .701 .094 
+5 -58 .60 12 46 .92 5 -686 —.223 · 
Tk 2.97 2.46 1.01 2.33 2.28 +,=11.05 

аһ .893 . 740 . 304 . 701 -686 Ха, =3.324 x .058 .533 

1 


737 ren 
m= . 300828 
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The theorem of equation (9) can be verified in Table 3. The sum of the 
first-factor loadings, with variables 1 and 2 reversed, ix 3.324, and (3.324)? — 
11.05, which is the sum of the coefficients in this table. The first-factor 
loadings in Table 4 represent all variables with positive sign as originally 
given. 

The first-factor residual coefficients can be written by equation (1), in the 
form 


" 
NS 
(18) Taje = Tjk алаа = Quim « 


The first-factor residuals are shown in Table 5. They were computed from 
the given coefficients in Table 2 and the first-factor loadings, first column, 


Table 5 


First-Factor Residuals 


+2 +3 E4 H5 

—.061 —.0590 — —.015 ‚032 
T .172 .164 .038 —.093 
+3 164 -158 .037 —.089 
Ф | .038 .037 .009 — —.021 
+5 | —.0903 —.089 —.021 ‚049 
Ch —.079 .222 212 .040 —.120 
х | —.080 ‚220 211 O18 —.122 
|$ 0 1598 507 120 284 
Sr, —.103 ‚048 053 ‚039 —.171 
Neg. 3 2 2 2 3 


of Table 4. The check sums, Ch, of Table 5 were obtained from the column 
sums X of Table 2 and the first column of Table 4. Example: —.13—(— .893) 
(+.058) = —.079. The algebraic column sums are shown in row ~ and the 
absolute column sums in row X| of Table 5. The sums, omitting diagonals, 
and the last row shows the number of negative co- 
efficients in each column. Ordinarily, all these rows are not computed, 
They are shown here to illustrate different methods of determining which 
variables to reflect. By the absolute sums we could reflect test 2. By the 
sums, omitting the diagonal, we could start by reflecting test 5; and, by 
counting the number of negative signs, we could reflect either test 1 or 


test 5. 

In Figure 3 we ha 
and second centroid axes. 
and its configuration is there 


are shown in row Ууз, 


ve the original configuration of five points and the first 
The correlation matrix in Table 5 is of rank 1, 
fore of dimensionality 1. The one-dimensional 
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residual configuration is shown in Figure 3, where the five test vectors are 
indicated by as many collinear arrows in the second centroid axis. These 


ЕН 


Ficure 3 


residual vectors are seen to be the projections of the five original test vectors 


on a one-dimensional subspace (axis ZZ) orthogonal to the first axis. When 
the residual vectors 1 and 5 are reflecte 


d, the resulting five vectors are uni- 
directional, and the second-factor lo 


adings are then equal to the residual 
vector lengths. The computation of the second-factor loadings is shown in 
Table 6, and they are recorded with proper sign in the second column of Ta- 
ble 4. The second-factor residuals vanish, since the given correlation matrix 
Table 6 


Pirst-Faclor Residuals after Sign Changes 


-1 T2 
— -1 | ооз б oak .015 — .032 
ү: .061 — 172 :164 038 “ona 
ЕЕ 059 — 14 158 037 ‘980 
+4 015 08%  :037  :009 091 
ur: 032 1093  .089 021 019 
ть 190 .58  .507  .190 284 
ар 149 144 1397 094 293 


т = .7835 
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of Table 2 is of rank 2 and the rank of each residual matrix is one less than 
the previous correlation matrix. 


The group centroid method 

The procedure to be described here is one of the simplest forms of the 
centroid method. 

In Table 7 we have an eight-variable correlation matrix. The last column 
shows the algebraic sum of each row, and it will be used for check purposes. 
The check column is vacant in the given correlation matrix. The first sum- 
mation row shows the absolute sum of each column. The highest absolute 
sum is 4.06, so that test 8 is chosen as a pivot test. In this column we choose 
several tests, in this case three, which have the highest correlations with 


Table ? 


First Factor Determined by Group Centroid Method 


3.20 3.20 


1.30 4.30 
1.045 1.044 


т 


= 16.95 


T=169 VT = 4.11 
Хаа = 4117 (119) 


test 8, disregarding sign. These correlations are пъ = +.67, ri = —.79, and 
Ts = +.86. In practice these subgroups may contain three to five tests or 
more, provided that only those tests are included in the subgroup which 
have significant correlations with the pivot test. 

The three tests in the subgroup are 5, 6, and 8, and the signs of their cor- 
relations with test 8 are +, —, and +, respectively. In row s we record unit 
weight with these signs for columns 5, 6, and 8, and zeros for the rest, as 
shown. 

In the next row we record the column sums for these three rows with signs. 
The first entry in row En; is obtained from the first column, namely, 
(4-.15) —(+.03)+(—.24) = —.12. The other entries are obtained in a sim- 
ilar manner. The check sum is obtained from the X column, namely, 
+ (+1.240) — (— 1.100) +(+.860) = +3.200, which is the expected sum. 
The actual sum of the row is the same, and it is recorded in the X column, 

Note, next, the highest absolute sum in this row, which is —2 As a 
practical, but somewhat arbitrary, rule, we note that one-third of this sum, 


Table S 


Second Factor Determined by Group Centroid Method 


cee 


— 004 « 
012 = 


m = 387579 
= 6.656 


Table 9 


T = 6.658 


Уаш 


2.580310 m = .387550 
(2.580)? = 6.656 


Table 10 
Factor Matrix by G troup Centroid Method 
I п III т 

1 —.182 .501 
2 .109 .939 
3 —.620 -570 
4 .440 .649 
5 -785 . 860 
6 —.880 .811 
7 471 890 
8 -921 


1081 `860 
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which is about .80, and all columns whose sums are less than this number 
are given a weight of zero in the next row, и, as shown. The other columns 
are given weights of +1 or —1 according to the signs of Xr; as shown. 
With the weights u we find the column sums ruz, which are recorded in 
the next row. The first entry in this row is obtained from the first column 
with the weights given in the row «w. It is —(+.19)+(—.53)+(+.15) 
(4.03) 4-(4-.09) 4-( —.24) = —.75. The check sum, +4.30, is obtained with 
similar weights from the X column, and it agrees with the actual sum of the 


entries in this row. 
Summing the entries in the row and applying again the same weights, we 
have 
+ 2.55 + 1.81 + 3.23 + 3.62 + 1.04 + 3.79 = 16.04 = T, 


-— 1 кє : З 
from which we obtain the multiplier ше aaa Applying this 


multiplier to row Xr, and to its sum 4.30, we have the factor loadings aj: 
with proper signs and their expected sum, 1.045. The actual sum of the row 
is 1.044, and it is recorded in the X column. Finally, the algebraic sum of 
the row with weights u is Ха 4.117, and the square of this sum is equal 
to the value of T or r, of equation (9a). The factor loadings in row ал are 
copied direetly in the first column of the factor matrix in Table 10. 

Table 8 shows similar computations for the second-factor loadings. The 
cheek column is obtained from the X column of Table 7, the first-factor 
loadings, and their sum, 1.045. The check sum in the first row of Table 8 
is obtained thus: +.460— (—.182)(1.045) = 4-.650. The actual sum of the 
residuals in the first row of Table 8 is 4.051, as recorded. The second-factor 
loadings, ау», are recorded in the second column of Table 10. 

Table 9 shows the corresponding computations for the third-factor load- 
ings. The third-factor residuals computed from Tables 7 and 10 vanish in 
the range + .001. | 

The computational routine for what we have called the “group centroid 
method” is so simple that a clerk can easily learn it. Attention should be 
called to the fact that the subgroup s should include three or more tests. 
If only one pivot test were used and if the weights u were made +1 or —1, 
omitting the zero weight, then this method would reduce to what was 
called the “simplified centroid method,” which was published in 1933.* 
That method was defective, in that the communalities of the pivot tests were 
sometimes even exceeding unity. Moreover, the method did not 
properly separate the common factors from the unique factors. The method 
which has been described here and which has been called the “group cen- 
troid method” avoids these difficulties by using the subgroups 8 and by using 
the three weights for u. However, the group centroid method is probably 


too high, 


* 4 Simplified Multiple Factor Method (University of Chicago Bookstore, 1933). 
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not so effective as the complete centroid method in separating the common- 
factor space from the unique test space. 


The complete centroid method 


The complete centroid method which has been in use in the Psychometric 
Laboratory for a number of years will be described here. This description 
will be given in terms of the computations on a standardized data sheet, 
which is illustrated in Tables 11-13 inclusive. The complete centroid 
method will be described with a numerical example of twelve variables, and 
we shall include here a description of all the numerical checks which consti- 
tute a part of our computational routine. The computational checks are 
explicitly shown on the data sheets. 

By the complete centroid method we mean the successive reflection of 
variables, one at a time, until all the column sums in the correlation matrix, 
or the residual matrix, are positive or zero. These column sums include the 
diagonal entries. No matter what the signs may be in a given correlation 
matrix, one can always reflect some of the variables so that all the column 
sums become positive or zero. In the complete centroid method this work 
is done systematically by reflecting one variable at a time. While it is al- 
ways possible to make all the column sums positive or zero by reflecting 
some of the variables, the factorial solution is, in general, not unique. If 
simple structure exists in the test configuration, the rotated factor matrix 
is unique. It is independent of the factor matrix that is obtained by the 
centroid method. The lack of uniqueness of the centroid method of factor- 
ing constitutes, then, no problem because the rotated factor matrix is unique 
and the structure is unambiguous. 

The following paragraphs are numbered to show the successive computa- 
tional steps in the complete centroid method with successive adjustment 


of the communalities: 

1. A given correlation matrix with twelve variables is recorded in Ta- 
ble 11. The diagonal entries are left blank, as shown. This example 
is intended to show the computational routine that is used on experi- 
mental data for the particular ease in which it is decided to adjust 
the diagonals after each factor has been extracted. 

The column sums are recorded in row s, and the row sums are re- 
corded in the column X. The sum of all the entries in the correlation 
matrix is 79.40, which is recorded at the intersection of row s and 


w 


column X. . : TNI" А 
3. Since all the given correlation coefficients are positive іп this exam- 


ple, we omit several steps that are required for subsequent factors, 
" 
Row s is copied into row C, as shown. 
4. The communalities are recorded in row d. Here the computer decides 
how to treat the unknown communalities. He may use one of the 
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estimation formulae for the communalities and record the estimates 
in this row. Several such formulae are described in chapter xiii. If 
he decides to use unit communalities, these may be recorded in 
row d. The simplest estimation formula is to take the highest co- 
efficient in each column as the estimated communality, and this is 
the method used in the present numerical example. The data sheet 
is designed so that it is applicable for any method of estimating or 
computing the diagonal values. Since there have been no sign 
changes in this table, row d is copied into row D, as shown in Table 
E 

5. Row E gives the sum of C+D. 

6. The entry 88.32 in row E is the sum of the entries C and D immedi- 

ately above it. Row E is then summed. If the sum agrees with the 

already recorded sum, this fact is indicated by a check mark, as 

shown. 

The first entry in the small table at the lower right corner is 88.32, 

which is obtained from row Æ. It is denoted T. 

8. The value VT is next recorded, as shown. 

9. The reciprocal m is recorded in the next space. 

10. The product mT is equivalent to 9.40 and is recorded in the next 


~ 


space. _ 

11. Next, it is verified that this value checks with VT. This verification 
is indicated by the check mark, as shown. 

12. The last row of the table shows the factor loadings, which are the 
products m4, as shown. Applying the same multiplier to the check 
sum gives MSK = 0.40, as shown. 

13. "The algebraie sum of the last row is 9.39, as recorded in the X col- 


umn. The fact that these two sums agree, namely, 9.39 and 9.40, is 


indicated by a check mark. 
14. The sum of the absolute values of the entries in the last row (9.39) is 
recorded in the small table. The fact that it agrees with the value 


of mT is shown by a check mark. 
15. The first-factor loadings are recorded in the first column of the factor 


matrix F in Table 15. 


The calculation of residuals 
The computations for the first-factor residuals and for the second-factor 
loadings are carried out on a new data sheet; as shown in Table 12. The suc- 


cessive steps in these computations are as follows: 


1. The first-factor loadings are recorded in column +a, as shown. The 
sum of this column (9.39) is also recorded. 
—a) the same factor loadings are recorded with 


2. In the top row (b= i 
f the top row is —v= —9.39, as recorded, 


reversed sign. The sum 0 


-164 


~ 


. The check sum (—.95 


- The residuals for the second variabl 
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The first-factor residuals (ру) are ordinarily expressed in the form 
Pik- Т — алар. 
Since the first-factor loadings, aj; and аз, may be positive or nega- 
tive and since their product is preceded by a minus sign, the com- 
puter is ordinarily required to keep in mind these three signs. The 


computational routine is simplified by expressing the residual in the 
form 


Pik = Tj + (aj)(—ag). 


The factor loading (ад) is represented in. the column (+a) and the 
factor loading (—ag,) is represented by the top row (b) on the data 
sheet. The actual computation of residuals will be illustrated for the 
first correlation coefficient, namely, rs = +.17 in Table 11. The resid- 
ual рә is to be recorded in the cell entry (21) in Table 12. The factor 
loadings .70 and —.59 are found in column a and row b directly op- 
posite the cell entry (21). 'The actual computation for this residual 
then is ' : 


17 + (70) (—.59) = —.24. 


The residuals in the rest of the column for the first variable are com- 
puted in the same manner, The diagonal cells are left blank in the 
residual table. 

5) for this column of residuals is next recorded. 


The value s is obtained from the previous data sheet. In detail the 
computation for this entry is 


4.94 + (9.39) (—.59) + (—.59)? = —.25. 


- The sum (—.23) of the first column of residuals is recorded in TOW s. 
. If this sum s agrees closely with the check sum Ch, as in the present 


case, it is assumed that the 


computations were done correctly. The 
slight discrepancy is to be 


expected from rounding. When the agree- 
ment of the sum and the check sum has been noted, the first column 
of residuals is copied into the first row of the table. The sum (s= 
—.23) is copied into column x. 


- The transcription in the first row is verified by actually summing that 


row. The agreement of the actual sum with the recorded sum is in- 
dicated by the check mark. 

е are next. computed, beginning 
in cell (32). When the residuals in the second column have been com- 


puted, they are verified in the rows Ch and s as before. The second 
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column of residuals is then copied into the second row and verified by 
adding the second row. The rest of the residuals are computed in the 
same manner. It should be noted that the number of residuals to be 
computed is reduced in the successive columns. When the computer 
reaches column 11, he has only one residual to compute. When the 
computer reaches column 12, there are no new residuals to compute. 
The last column is verified by the check sums. 

9. The final check for the whole table of residuals is obtained in the 


form 
Ss — XD. 


Its numerical value ( — 1.32) is recorded as shown. 

10. The sum (— 1.30) of the row Ch is next recorded. The fact that this 
sum (— 1.30) agrees closely with the previously recorded sum (— 1.32) 
is represented by a check mark in the small square.* This verifies the 
computations in the row Ch. | 

11. The sum (—1.28) of the row s is next recorded. The fact that this 
sum agrees closely with the two sums just recorded is represented by 
a check mark, as shown in the second small square. 

12. The entries in column 2 are summed. The fact that this sum agrees 
with the already recorded sum (—1.28) is represented by a check 


mark adjacent to this sum. 


The sign changes n : 
1. If all the entries in row s are positive, then no reflections are necessary. 
This was the case in Table 11. However, in Table 12 TOW s contains 
negative entries. Hence we proceed by recording the entries in row А. 
The entries in row A are recorded to three decimals. The previous cal- 
latina have been: recorded to two decimals. The anticipated sum 
(.640) is recorded, as shown. Row А is ext Kien i » fant that 
the sum agrees with the expected sum ie eg by n. êk k. 
The highest positive number in row A is. | i i 115 а à en 
in parentheses, and it occurs 1n the seconc еш h us act 18 rep- 

resented by writing +(2) at the beginning о а ле next row. | 
of the corresponding cell entry of 


; ; RE TUS sum 
3. Eac v in this row is the $ d : 
s Pris second row of residuals. For example, the first entry is 

row A anc 8 


(115) + (—.2®) = —.125. 


N 


А 1 e calculations the diagonal entries are treated as 
In the sign-chang in row +(2) is the same as the second 


zeros. Thus the second entry 


1 data sheet are represented here as parentheses. 
ted data sheet = 


arp : а 
Phe boxes in the prin 


166 


6. 


- The highest positive sum in this row is 
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entry in row A. Each entry in this row is written in parentheses if the 
corresponding entry in row A is in parentheses. 


- The anticipated sum (.400) is obtained from the corresponding en- 


tries in column Х, namely, .640 and —.24. The sum of the new row 
agrees with the anticipated sum, .400. This fact is represented by the 
check mark. 


, and hence this number is 
written in parentheses. This number is in column 9, and therefore the 
next row is denoted +(9) as shown. Each entry in row +(9) is the 
sum of the corresponding entries in row +(2) and in the ninth row of 
residuals. For example, the first entry is 


(—.125) + (—.19) = —.315. 


The sums in row +(9) are written in parentheses if the corresponding 
entries in the previous rows are in parentheses. It will be scen that 
the entries in these columns are written in parentheses after the sign 
change for that column. The entries in row +(9) are summed and 
checked as before. This procedure continues until all the entries in a 
row are negative, zero, or positive in parentheses, 

It happens occasionally, especially with a large test battery, that a 
test vector is reflected twice. In determining the next variable which 
is to be reversed, we choose either the largest positive number without 
parentheses or the largest negative number in parentheses, whichever 
has the larger absolute value. The same result is obtained if we regard 
the parentheses as the equivalent of a negative sign and if the rule is 
adopted of reversing that variable which has the largest positive value. 
If, according to the rule, a variable must be reversed which has previ- 
ously been reversed, this can be accomplished by subtracting the cor- 
responding row of residuals instead of adding, as in the regular pro- 
cedure. 


When all the entries in a row 
as the equivalent of a neg; 
Table 12. 


are zero or negative, counting parentheses 
ative sign, this row is denoted В, as shown in 


. The entries in row C are recorded to two decimals. 
. The weighted sum of the twelve entries in column X is recorded in 


row C. Each weight is either +1 or — 1. The weight is +1 if the vari- 
able has not been reversed, and it is —1 if the variable has been ro- 
versed. "The weighted sum of the twelve entries in column X is then 
—.12, as shown. 


- The entries in row C are summed. The sum must equal the antici- 


pated sum, which here is —.12. 


This agreement is represented by the 
check mark, 
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The communalities 

1. In row d are recorded the estimated diagonals. Each entry in row d 
is the largest absolute value of the residuals. For example, in the 
second column the residual with largest absolute value is —.24, It is 
recorded in row d with positive sign. 
In row D the same values are recorded with the signs indicated in 
row C. The weight, и, is +1 for unreflected test vectors and —1 for 
reflected vectors. The sum (.18) of row D is recorded as shown. 


Ww 


Calculation of factor loadings 

1. Row Æ is the sum of entries in rows C and D. The anticipated sum 
(.06) is, similarly, the sum of X-entries of rows C and D. Row E is 
summed. The fact that the sum of row Æ is equal to the anticipated 
sum (.06) is indicated by the check mark. 
The sum of the absolute values in row Æ is 10.60, which is recorded in 
the small table. It will be noted that X 
3. The value of V7 is next recorded in the small table. 
4. The reciprocal m is recorded in the next space in the small table. 
5. The product mT is recorded in the next space of the small table. It is 
recorded to two decimals. Its agreement with VT is indicated by the 


w 


7| = Ур. 


check mark. 

The factor loadings are recorded in the last row of Table 12. The an- 

ticipated sum, тх = .02, is recorded іп the check column. 

The sum of the factor loadings (.03) is recorded in the X column. The 

fact that these two sums agree closely, namely, .03 and .02, is indi- 

cated by a cheek mark in the small square. P 

8. The sum of the absolute values (3.27) of the factor loadings is recorded 
in the last space of the small table. The fact that this sum agrees 


closely with mT is indicated by a check mark in the small square. 


The same routine is followed in computing the second-factor residuals, 
Which are shown in Table 13. The third-factor residuals are shown in Table 
14. These residuals are too small to justify further factoring. 97 

The factor loadings are assembled in the factor matrix F of Table 15. The 
fourth column of this matrix shows the communalities as paa pre by the 
centroid method, with successive adjustment in ш cells. The last 
column of this table shows the true communalities, x a a tih Pw ent with 
the side correlations of the given correlation ides Ww а | Е rank 3. 

When the true communalities are known, as m t A Lope ; titious axe 
ample, the same data sheets can be used for ien E e Ses a 
Way that the third-factor residuals vanish identica oe * ms ane ! com- 
munalities are then exactly equal tot he true E E | ер edure 
is then to record the true communalities m d is m “i 8 i computing 
the residuals for Table 12 the communalities are trea ame way as 


~ 


OT 


А | 


Table 14 


b=-a —.29 .03 д2 —.20 Er —.0 .09 .02 —46  —42 21 —v--.01 
+a Test 1 2 3 4 5 6 7 8 9 10 11 12 z Ch 
:29 1 —.01 .01 .00 .01 .01 .00 .02 .00 .01 .01 .00 .06 у 
—.03 2 —.01 .01 .01 .02 -00 00 .01 .00 .00 .00 .00 02 vy 
— .26 3 Ol —.01 00 —.02 .01 .01 .00 —.01 .00 .00 .01 00 y 
—.12 4 .00 .01 .00 .00 .00 01 —.01 .01 .00 .00 —.01 .01 м 
.20 5 .01 .02 .02 .00 .02 401 —.01 .01 —.02 .00 .00 02 y 
—.13 6 .01 .00 .01 .00 .02 —.01 .01 .00 .02 .01 .00 JT чи 
.10 7 ‚00 .00 .01 .01 .001 —.01 .01 .00 .00 .00 .01 4 X 
—.09| 8 .02 .01 .00 —.01 —.01 .01 .01 ‚01 —.01 01  —.01 . 0: у 
—02, 9 ‚00 00 .01 .01 .01 .00 .00 .01 .00 .00 .01 03 y 
.16 | 10 .01 .00 .00 .00 —.02 .02 00 —.01 .00 .00 .00 00 у 
412:| 1T ‚01 ‚00 .00 .00 .00 .01 .00 .01 .00 .00 .01 04 vy 
£821 12 .00 .00 „01 ~B .00 .00 01  —.01 .01 .00 ‚01 02 vy 
Ol=v= a 
sub +b? =Ch .05 .02 .01 .03 .02 .06 .05 .02 .03 .02 .03 .03 85.35 9Xs—vt- xb 
Er-s .06 .02 .00 .01 .02 .07 .04 .03 .03 .00 .02 .02 .94 


РУСА (V)+ zzi (V) 


Table 13 


Sor 


b=-a | –.45 49 —.1 —.02 BL E! 40 =I Al —.02 a —.08 
1 2 3 4 5 6 7 9 10 11 12 3 Ch 
—.07 —.03 °з -.01 06 ۷ 
= .02 00 01 00 200 00 v 
—07 00 -03 —02 00 —о4 y 
— :03 101 .03 00 .0  —.02 0 
07 01—07 —.02 n 01 01 К; 
= 08 00 104 202 EN: 200 00 y 
:03 фо —02 :00 00 102 y 
= 01 01 102 100 .00 .01 = 102 Y 
=.01 00 00 01 200 En 100 Š 
206 00 —.04 -.02 02° —02 .00 M 
«0 —.03 —01 101 00 00 .02 У 
— :06 01 206 2 — 01 :01 .01 = 03 .02 a 
01 —.08 00 02 02 0% 0l —.04 —.0 —.H =u: 
02 —.08 E .04 04 00 02° —02 о? УСА (s) XSr (V) 
—.010  (.040) — .005 —.030 —.020 :000 —.010 010 = “010 2 
—.010 (040) 020 —.010 —.040 —.040  (.070) — 090 
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the correlation coefficients. The residual communalities are then recorded 
in row d of Table 12 without any adjustment. When this procedure is fol- 
lowed, the third-factor residuals vanish identically, including the commu- 
nality residuals, and the obtained communalities are equal to the true com- 
munalities. This procedure would be used with experimental data except 
for the fact that the true communalities are unknown and hence 
estimated. In dealing with experimental data the residu 
are adjusted after each factor has been extr 


must be 
al communalities 
acted in the manner which has 


heen described. The computer may prefer some other method of adjusting 


Table 15 


OnTAINED True 
VALUES 
1 I ш D: D 
1 .59 .45 .29 7 
2 .70 —.49 —.03 ‚74 
3 .80 .21 —.26 JT 
4 .90 .02 —.12 .82 
5 .80 122 .20 .72 
6 .80 = .31 —.13 ‚74 
7 .80 —.81 .10 74 
5 .80 ise —.09 74 
9 ‚70 —.40 —.02 .65 
10 .70 Al .16 66 
11 .90 —.11 319 .83 
12 .90 .02 —.21 .85 
x 9.39 .03 .01 


the estimate of the communalitie: 


S for each residual table by using one of 
the estimation formulae in chapte 


FFU 
A multiple group method of factoring the correlation matrix 


Most of the methods of factoring the correlation matrix require the cal- 
culation of residuals after each factor has been extracted. This is perhaps 
the most laborious part of factoring. The method to be described here 
avoids the computation of residuals after each factor has been computed. 
Since the method turns on the selection of groups of test vectors, it will be 
called a multiple group method of factoring.* The method can be used for ex- 
tracting one factor at a time if that is desired; but it will be considered here 
for the more interesting case in which several groups of tests are selected 
from the correlation matrix at the Start. The result of this method of fac- 
toring is a factor matrix F which satisfies the fundamental relation FF’ = R. 
The multiple group method of factoring is general, in that it can be used 


ыў I+ Thurstone, “A Multiple Group Method of Factoring the Correlation Matrix," 
Psychometrika, X (June, 1945), 73-78. 
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on a correlation matrix of any rank, any order, and any configuration of test 
vectors, and the method is successful for any arbitrary grouping of the 
tests, provided that the several groups of tests are linearly independent.* 
This method of factoring has a limitation, in that the communalities are as- 
sumed to be known at the start. This is not a serious hardship, because the 
factoring can be repeated with the communalities that are found in the first 
trial and the convergence toward the correct communalities can be expected 
to be rapid. 

This method of factoring is based on the well-known principle that a 
common-factor space of r dimensions is spanned by any set of linearly in- 
dependent vectors in that space. The group centroid vectors are used for 
this purpose. 

The method of selecting the groups of tests is not crucial for the multiple 
group method of factoring, since the only requirement is that the centroid 
vectors for the several groups shall be linearly independent. This condition 
is nearly always satisfied without taking any special precautions about it. 
With experimental data it would be a rare cs in which this condition 
would not be satisfied. If, by chance, the centroid vectors for the several 
groups should happen to be linearly dependent, then this fact would become 
apparent in a later step of the calculations in which an inverse is computed. 

In selecting the several groups of tests for the purpose of factoring, the 
computer will naturally put together the tests that have high intercorrela- 
tions. He will select, for each group, those tests which are nearly collinear, 
if such groups can be found in the correlation matrix. If such groups are not 
readily seen by inspection, then the grouping can be carried out by some 
other routine. One such routine is as follows: Select, as the first pivot, a 
test which has one or more significant correlations and which also has an 
appreciable number of low coefficients, if these can be found. Arrange as г 
group those tests that have significant correlations with the first pivot test, 
either positive or negative. Retain, as the first group, those tests which, by 
reflection if necessary, produce a subgroup in which all correlations are posi- 
tive or zero. Choose the next pivot test from the remaining tests in the 
same manner, Proceed in this way until all the tests with significant corre- 
lations are represented in the several groups. The grouping of the tests can 


* Holzinger has described a factoring method which anticipated the first two parts of 
the multiple group method of factoring, namely, the computation of an oblique factor 
matrix V by summations and the computation of the correlations bet ween the group cen- 
troid axes (“А Simple Method of Factor Analysis,” Psychometrika, Vol. IX (December, 
1944]). The multiple group method adds a third step, namely, the computation of an 
orthogonal factor matrix Ё, which is the starting-point for the rotational problem. In de- 
scribing his factoring method, Holzinger limits the method to correlation matrices which 
can be divided into sections of unit rank. However, his method is general, and it сап be 
used on correlation matrices of any order, rank, and configuration. His method is alge- 
braically equivalent to the first two steps in the multiple group method. 
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be carried out in any one of several ways for the multiple group method of 
factoring. 

In Table 16 we have a numerical example which illustrates the multiple 
group method. The given correlation matrix has eight variables with inter- 
correlations as shown. The communalities were known in this case. With 
experimental data the communalities must be estimated. In the present ex- 
ample, the grouping of the tests was quite arbitrary, with little reference 
to the correlations. Tests 1, 2, 6, and 7 were placed in group 1, and tests 
3, 4, 5, and 8 were placed in group 2. Only two groups were used in this 
example. The natural procedure is to group the tests into correlated clus- 
ters as far as possible. The multiple group method is not dependent on the 
grouping, and hence the groups were selected arbitrarily for this numerical 
example. 

In row S, we have the sums of columns for rows 1, 2, 6, and 7. In row Ss 
we have the corresponding sums for rows 3, 4, 5, and 8. The correlation 
matrix has now been augmented by two rows, but the rank is unchanged 
because these two rows are linear combinations of the given eight rows. If 
two similar columns were added to the right of the correlation matrix, we 
should have an augmented correlation matrix of the same rank. This has 
not been done in Table 16 because it would simply be the transpose of the 
two rows of matrix S, which are already computed and recorded in the table. 

The sum of the corresponding entries in each row of the matrix S are 
then computed and recorded, as shown in matrix T. For example, the entry 
1.28 is the sum of columns 1, 2, 6, and 7, of the first row of S. The other 
sums are computed in the same manner. The square matrix T is of order 
2X2, and it represents the lower right corner of the augmented correlation 


matrix in which the rank is unaltered. А 
ompute the reciprocals of the square roots of the di- 


The next step is to ¢ 1 [ 
atrix T. These are the weights wı and we, which 


agonal elements of the m 
are shown to the right of the matrix T. | | 

The matrix V’ is computed by applying the weight wı to the first row 
of S, and the weight ws to the second row of S. Similarly, the same went 
on the rows of T give the square matrix U, as shown. The matrices V’ and 
ht of as two rows by which the correlation matrix could 


U may be thoug 
уы The matrix U would then be 


be augmented instead of the rows of S and T. Uw | 
the lower right corner of the augmented correlation matrix, in which the 
rank is still unaltered. х 

The two weights, Wı and we, are applied to the columns = the ma- 
trix U, and we then have the matrix A, with unit diagonals. The calcula- 
art checked by the unit diagonals of this matrix. This matrix 
angular separation of the two unit reference vectors 
Each one of them contains the centroid of a group 


tions are in p 
shows the cosine of the 
that have been chosen. 
of test vectors. 


174 MULTIPLE-FACTOR ANALYSIS 


If we write the transpose of the matrix V’, we have an oblique factor ma- 
trix V, as in previous methods of rotation from the orthogonal factor ma- 
trix F, but we have obtained here the oblique matrix directly and without 
computing the orthogonal factor matrix F, which is usually obtained first 
by the various factoring methods. 

Since we have here an oblique factor matrix V, as well as the cosines of 
the angular separations of the unit reference vectors for the factor matrix V, 
we ean determine the transformation which should carry the oblique factor 
matrix V into an orthogonal factor matrix F. Since the location of an or- 
thogonal frame in the system defined by V and U is not unique, we must 
first define in some convenient way the location of the desired orthogonal 
reference axes of F. We shall do this by locating the first of the orthogonal 
references axes collinear with the first of the oblique axes. Then the first 
column of F will be the same as the first column of V. The next orthogonal 
reference axis will be located in the plane of the first two oblique axes and 
orthogonal to the first one. This process is, in fact, the diagonal method of 
factoring applied to the correlation matrix Rpa. When this is carried out 
routinely, we have the factor matrix /,,, in which the two oblique ref- 
erence axes p are defined in the orthogonal frame m. 

The inverse of the transpose of this matrix gives Аш = (hn) !. We can 
now write the transformation by which the orthogonal factor matrix F can 
be written for the whole test battery. The matrix Fg, then is the additional 


two rows of F which describe the oblique reference axes р. This transfor- 
mation is 


(19) Vat =F, 


from which we have 
(20) PA y. 


where A is already numerically given. 

Applying this transformation to the matrix V by equation (19), we ob- 
tain the factor matrix F, which is shown in the upper right corner of 
Table 16. 

When this factor matrix has been written, a residual table can be com- 
puted for the two factors, to determine whether additional factors need to 
be extracted. In the present example, which is exact, the residuals vanish 
identically. In a practical problem with experimental data, the residual 
table should be computed and inspected to determine whether additional 
factors should be extracted. If that is desirable, one proceeds with the 
residual table as with the original correlation matrix in the present example. 
Groups are then listed from the residual correlation matrix until the tests 
with significant residuals seem to be represented. The process is repeated 
so as to add one or more columns to the factor matrix F. 


THE CENTROID METHOD OF FACTORING 175 


In the present case we have one set of residuals to compute for the entire 
factor matrix Ё instead of several sets of residuals, as would have been re- 
quired in most of the current methods of factoring the correlation matrix. 
It will also be seen that the order of the inverse to be computed is equal to 
that of the number of factors determined by this process. For example, if 
seven factors are to be determined and if the process starts with seven linear- 
ly independent clusters, then there is an inverse of order 7X7 to be com- 
puted. If the factoring proceeds with, say, only four groups, then there is 
an inverse of order 4X4 to be computed, and the fourth factor residuals 
should then reveal three additional groups, resulting in three columns to be 
added to the factor matrix F. The computational labor is probably mini- 
mized by finding as many groups in the given correlation matrix as there 
are common-factor dimensions, so that only one table of correlational 
residuals needs to be computed. However, if the first estimate of the num- 
ber of clusters is too small, one merely repeats the process until the residuals 
vanish. If the factoring is done in several steps, the inverses are, of course, 
of smaller order. 

The grouping was chosen quite arbitrarily in this example in order to il- 
lustrate in a numerical example that the multiple group method of factoring 
is independent of the method of grouping the variables. The only require- 
ment is that the centroid vectors of the several groups must be linearly in- 
dependent. 

It is not necessary to select the same number of tests in each group, but 
there should be at least three or four tests in each group. This requirement 
is associated with the separation of the common factors from the unique 
factors, A limiting case is that in which each group contains only one test. 
The multiple group method then degenerates into the diagonal method of 
factoring.* The multiple group method is not quite foolproof. A computer 
might assemble a group of tests whose sum would give a null vector. It 
would then be necessary merely to alter the grouping. 


* L. L. Thurstone, Theory of Multiple Factors (Ann Arbor, Mich.: Edwards Bros., 
January, 1933), pp. 13-16. 


CHAPTER IX 
CONFIGURATIONS AND FACTOR PATTERNS 
Interpretation of factorial structure 


We have seen that every correlation matrix implies a configuration of test 
vectors and that the number of dimensions of the configuration is the rank 
of the correlation matrix. It has also been shown that a reference frame 
must be inserted in the configuration before a factor matrix can be written. 
The combination of the configuration and the reference frame is called a 
“structure.” The problem to be considered now is how to locate the ref- 
erence axes in relation to the configuration so as to give the most plausible 
interpretation for the reference frame. All students of the multiple-factor 
problem agree that the reference axes should be placed in some relation to 
the configuration, but they differ on the criteria by which this should be ac- 
complished. These differences, which are the subject of considerable con- 
troversy, are due principally to the different purposes of factor analysis 
which the several writers have had in mind. What is correct and acceptable 
for one purpose may be irrelevant for another purpose. 

At the outset of a factor analysis, the investigator must decide whether 
he will write a factor matrix to account for the complete variance of each 
test or only the common-factor variance. His choice will depend on his 
purpose. If he decides to account factorially for the complete variance of 
each test, then he writes unity in the diagonal cells of the correlation matrix. 
The several methods of factoring are applicable in this case, just as well as 
in the case where communalities are inserted in the diagonal cells. The fac- 
tor matrix will then represent a set of n unit test vectors, which define the 
total test space. This is a subspace of the total factor space, which involves, 
in general, (n+r) factors or dimensions. The configuration will not involve 
any more dimensions or factors than there are tests in the battery, and it 
will reproduce the intercorrelations exactly, as well as the diagonal unit 
self-correlations. If the object is merely to write a factor matrix which re- 
produces the correlations, then the task is easily done by any one of several 
factoring methods. The reservation about this solution is that it is difficult 
to identify the factors, common and unique, from such a factor matrix; but 
that may not be the purpose of the investigator who proceeds in this way. 

If the purpose of a factorial analysis is to identify the functions or param- 
eters that produce the intercorrelations, then the problem can sometimes 
be solved by limiting the analysis to the common factors which are deter- 
mined by the reduced correlation matrix. If this is the purpose of the anal- 
ysis, the communalities are inserted in the diagonal cells of the correlation 

176 
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matrix. If there is reason to suspect that there is no unique variance in the 
measurements, then unity сап be used in the diagonal; but such situations 
are rare in those fields in which factor analysis is likely to be most useful. 
With the communalities in the diagonal, we have the possibility that the 
number of common factors is considerably less than the number of measure- 
ments, so that the problem becomes soluble, at least as far as the number 
of unknown common factors is concerned. 

The purpose of a factor analysis also determines how the data are to be 
treated, even after a decision has been made concerning the self-correla- 
tions. The two conflicting purposes that are largely responsible for contro- 
in this field are (1) the statistical condensation of the correlations into 
a factor matrix with maximum effectiveness and (2) the interpretation of 
the factors which represent distinguishable functions or parameters. Ever 
since multiple-factor analysis was introduced, it has been known that these 
variant solutions are mathematically equivalent. The debate is concerned 
with the choice of reference frame for the factor matrix, which is the end- 
product of the computational work and the starting-point for interpretation. 
These two purposes may overlap when an investigator tries to accomplish 
both the maximum statistical condensation of the correlation coefficients 
into a factor matrix and also a physical interpretation of the factors as rep- 
resenting fundamental concepts or functions. Some students of factor anal- 
ysis deny that meaningful factors exist in psychological measurements or 
performances, at least when they are revealed factorially. In other contexts 
the same authors speak confidently about “auditory acuity,” “mechanical 
aptitude,” “fluency,” “visualizing,” and many other components of abilities 
and disabilities; but, when these same functions appear in a factor analysis, 
ard the psychological interpretations as foolish. For such 


they promptly dise: | pretati 
authors an attempt factorially to identify abilities is supposed to be a for- 


bidden faculty psychology. 


Statistical criteria for the reference frame 

If the principal purpose of a factorial study is to condense the intercorre- 
lations into a factor matrix with a set of numbers that will account for the 
correlations, then we have a legitimate statistical problem that can be 
solved. In general, the factorial solution would be considered the more suc- 
cessful, the smaller the number of factors that account for the correlations 
with negligible residuals. The statistical condensation is judged to be suc- 
cessful if the number of independent parameters in the factor matrix is very 
much smaller than the number of independent correlation coefficients. By 
the usual statistical considerations the most accept able factor matrix would 
then represent a reference frame that is called the principal axes of the con- 
figuration of test vectors. It is located by a maximum variance criterion. 
Some of the characteristics of the principal-axes solution will be mentioned 


178 MULTIPLE-FACTOR ANALYSIS 


here. The first axis is so placed in the configuration that the sum of the 
squares of the projections of the test vectors on the first axis is maximized. 
There usually exists a unique solution for a given test battery. When the 
first axis is so placed, the first-factor products account for more of the vari- 
ances of the т tests than can be accounted for by any other axis in the con- 
figuration. When the same criterion is applied to the first-factor residual 
coefficients, the second principal axis is so placed that it accounts for more 
of the residuals than any other axis in the residual configuration. The 
first two factors, then, account for more of the test variances than any other 
pair of axes in the test configuration. For any given number of factors so 
extracted from the correlation matrix, the residuals are minimized, This is 
an interesting and important feature of the principal axes of a test configu- 
ration. This reasoning may be applied to a correlation matrix with unity 
or with communalities in the diagonal cells. 

When the correlation matrix is large, say forty or fifty tests, the prin- 
cipal-axes solution is quite laborious, and it is usually advisable to make 
some compromise. Another statistical principle that can be used for locat- 
ing the reference frame is to place the first axis through the centroid of the 
termini of the test vectors for the whole battery. This axis is usually nearly 
the same as the first principal axis when the correlation matrix has coeffi- 
cients that are all positive or zero. This type of solution can also be made 
either for the total test space or for the common-factor space. With some 
modifications, the subsequent axes can be placed so as to accomplish es- 
sentially the same purpose for the residual coefficients after the extraction 
of each factor. This principle does not lead to a unique solution for any 
given test battery. Its purpose is essentially similar to that of the principal- 
axes solution, in that one attempts to account for as much as possible of the 
test variances by each successive factor; but the centroid solution is not 
unique for any given test battery, and the solution does not have the in- 
teresting mathematical properties of the principal-axes solution. The cen- 
troid method of factoring and the centroid solution for the location of the 
reference axes are to be regarded as a computational compromise, in that 
they have been found to involve much less labor than the principal-axes 
solution. The residuals after the extraction of a given number of factors are 
somewhat larger than the residuals remaining after the same number of 
factors determined by the principal axes. 

The two principles that have been described for locating the reference 
frame are both dependent on the whole configuration. Each successive fac- 
tor is determined by some criterion that is applied to the whole test con- 
figuration. These criteria we have called “statistical” in contrast with the 
configurational criteria for locating the reference frame which depend on the 
nature of the test configuration as determined by its distinguishable parts. 

When we apply statistical criteria for locating the reference frame, our 
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principal interest is in the size of the residuals, which are made as small as 
possible. Less interest is then ordinarily given to the factor loadings them- 
selves, although some writers make attempts to interpret the factorial axes 
located by these criteria. When the successive factorial axes have been lo- 
cated with the purpose of minimizing the residuals, the factor loadings 
are regarded more or less as regression coefficients, whose principal useful- 
ness is that they account for the dependent variable with the smallest 
possible residuals. In most cases when a regression equation is written 
the principal concern is to predict the dependent variable with the small- 
est possible error, and little attention is then given to the interpreta- 
tion of the regression coefficients themselves. They serve ordinarily as a set 
of useful, but otherwise meaningless, numbers that do predict the dependent 
variable. In factor analysis we have no differentiation between independent 
and dependent variables, but we can apply statistical criteria to ascertain 
whether the several factors do account for the intercorrelations with mini- 
mum residuals. If this is the principal purpose of a factorial study, then it 
seems reasonable to expect the writer to show that he gains something which 


assical statistical methods involving multiple corre- 


is not available in the cl 
lation and regression equations. 

The statistical and the configurational criteria for locating a reference 
frame in the configuration of test vectors can be combined in what seems 
to be the ideal factorial study. If we disregard computational labor, which 
may be reduced by improvements in computational methods, the ideal solu- 
tion for a factor problem seems to be, first, to determine a set of r factors 
for the given correlation matrix which are so determined that the rth residu- 
al coefficients satisfy two requirements, namely, (1) that the residuals are 
the smallest possible for 7 orthogonal factors and (2) that they are small 
enough to be ignored. "This implies the first principal axes of the test con- 
figuration. When the solution has arrived at this point, we can be certain 
that the r factors account for more of the common-factor variances than 
can be accounted for by any other set of r orthogonal factors. The faetor- 
ing should be made with communalities in the diagonal cells of the correla- 
tion matrix. The next step in the solution is to find a reference frame in the 
r-dimensional configuration which will represent the distinguishable func- 
tions or parameters of the tests and their correlations. When such a refer- 
ence frame can be found, it represents primary factors in the test variances. 
There is no guarantee beforehand that such a reference frame can be 
found. If the test battery is a hodgepodge of complex tests, there may not 
exist in the configuration any clues as to what the underlying functions may 
be; but this possibility is dependent on the intuitions of the experimenter 
in assembling a significant set of measurements rather than on factorial 


theory and method as such. 
In this text it is assumed that the principal purpose of factor analysis is 
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to identify one or more underlying factors, functional unities, or parameters 
that produce the differential performances or other individual differences, 
even if the discovery of such a factor or factors should account for only a 
part of the test variances in any particular test battery, leaving considerable 
parts of the variances still unaccounted for and to be identified by future 
investigation. 

If it should be found, for example, that the ability for serial learning is 
distinct from the ability to memorize paired associates, then such a finding 
would be an important addition to our knowledge about the faculty or facul- 
ties of memory, even if the serial-learning factor should account for only a 
part of the variances of the memory tests involved. If it should be found 
that serial learning and paired associates do not separate factorially, then 
we should conclude that, as far as we ean as iin now, these two forms of 
recall are mediated by the same cortical functions. The question of whether 
these factors account for all, or only a part, of the variances of the particu- 
lar tests that we happen to use is, then, a secondary matter for the scien- 
tifie problem. At the same time, we should regard any unknown residual 
variance as a challenge to further exploration either by factorial or by other 
experimental means. 


The parameters or functions in terms of which a domain is described are 
probably never unique, and this corresponds to the fact that no one has 
ever demonstrated any factorial description to be unique. The best that 
can be said for any factorial description of a domain is that it reveals some 
aspects of an underlying order; but, in so far as this i: successfully done, it is 
likely to reveal alternative sets of parameters or functions that might be 
chosen for the scientific des ription. 
of the cylinders in chapter ii, this 1 
tion is easily seen, but it does not 
tions. The choice of fundament 
will be seen in the higher-dimer 
of taking a central or average f 


In the simple two-dimensional example 
ack of uniqueness of the factorial resolu- 
invalidate either of the factorial deserip- 
al axes would depend on the context, but it 
asional problems that the common practice 
actor for all the measurements in any particu- 
lar factorial experiment as one of the basie parameters for the domain tends 
usually to blur rather than to clarify the underlying order. The lack of 
uniqueness of factorial resolutions is in agreement with the principle that a 
scientific hypothesis can never be demonstrated to be correct; it can only be 
shown to be plausible for the data at hand or inconsistent with them. 

In the development of multiple-factor analysis one naturally raises the 
question as to whether the object is to account factorially for the original 
raw test scores or the intercorrelations. "These two objectives 
same. One might reproduce the intercorrel 
factor matrix which would not. enable us to reproduce the raw scores. Some- 
times this question is diseussed as an issue in which we might make a choice. 
To analyze a set of raw scores so that they are reproduced factorially in- 


are not the 
ations with communalities by a 
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volves more factors than can be solved for by present methods. Limiting 
the analysis to the common factors sometimes makes the problem soluble. 
By identifying one factor at a time and by building new test batteries for 
the exploration of unknown factors, we hope eventually to know enough 
about the factors involved so that the actual test performances themselves 
may be accounted for to an acceptable degree of approximation. That is the 
ultimate objective. Limiting ourselves to the common factors in each study 
is a means of eventually attaining the interpretation of the individual test 
performances. 

Considering factor analysis as a scientific method rather than as an effi- 
cient method of statistical condensation, we shall discuss different types of 
factors, configurations, and factor patterns to determine how they may be 
used in the discovery of an underlying order in the complexities of human 
traits or in other fields, where basic concepts may be formulated in terms 


of exploratory factorial investigation. 


Simple structure 

One of the turning-points in the solution of the multiple-factor problem 
is the concept of “simple structure.” It will be shown that this concept 
enables us to obtain an invariance of factorial description that has not, so 
far, been available by other means. The combination of a test configuration 
and the co-ordinate axes is called a structure. The co-ordinate axes deter- 
mine the co-ordinate planes. If each test vector is in one or more of the 
co-ordinate planes, then the combination of the configuration and the co- 
ordinate axes is called a simple structure. The corresponding factor pattern 
will then have one or more zero entries in each row. If a test vector lies in 
one of the co-ordinate planes in a three-dimensional configuration, then it 
can be deseribed as a linear combination of two co-ordinate vectors, so that 
one of its factor loadings is zero. If a test vector lies in two of the co-ordi- 
nate planes in a three-dimensional problem, then it is collinear with one of 
the co-ordinate axes, and it will have two zero factor loadings. If a test 
vector lies in all three co-ordinate planes in a three-dimensional problem, 
then it is a null vector with three zero factor loadings, and it is of no scien- 
tific interest. : ; 

When a factor matrix reveals one or more zeros 1п each row, we can infer 
that each of the tests does not involve all the common factors that are re- 
quired to account for the intercorrelations of the battery as a whole. This 
is the principal characteris ic of a simple structure. а tere dr and 
its corresponding factor pattern reveal that the compi exities 2 the individu- 
Al tests are lower than the complexity of the battery as à whole. When a 
configuration reveals that the test vectors lie in restricted subspaces, such 
as the co-ordinate planes, and in their intersections of ee Dy er dimension- 
ality, this circumstance can be used for the location of a reference frame 
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that will generally give a simpler interpretation for the individual tests than 
if each test is supposed to involve all the factors of the whole battery. Most 
of the configurations and factor patterns in the following section are ex- 
amples of simple structure, in that generally the rows of the factor matrices 
have one or more zeros. The degrees of convincingness and overdetermina- 
tion of simple structure and the algebraic relations will be discussed in sepa- 
rate sections. 


Types of factors 

Factors are classified in several types according to the manner in which 
they enter into the test variances of a battery. A factor which is of a given 
type in one battery may be of another type when it enters into the tests of 
another battery. The classification refers, then, to the manner in which a 
factor is involved in a given test battery. 

A factor which is involved in the variances of two or more tests of a bat- 
tery is called a common factor in that battery. Our use of the term “сот- 
mon factor” in this text implies that the factor is determined by the side 
correlations. We distinguish between common factors so determined and 
the unique factors determined by the diagonal self-correlations. Excep- 
tions to this use of the term will be explicitly noted. 

A factor which is involved in the variance of only one test of a battery is 
a unique factor in that battery. 

A common factor which is involved in the variances of all tests in a bat- 
tery is a general factor in that battery. 

A common factor which is involved in the variances of two or more, but 


not in all, of the tests of a battery is sometimes called a group factor in that 
battery. 


A common factor which is inv 
a battery is called a doublet. 

A common factor which is involved in the variances of only three tests 
in a battery is called a triplet. 


A factor which enters into the test variances of a battery with both posi- 
tive and negative signs is a bipolar factor. 


olved in the variances of only two tests in 


Types of configurations 


All the configurations discussed in this section are three-dimensional. 


The principles illustrated here can be generalized to configurations in n 
dimensions. 


Random configuration 


In Figure 1 we have three orthogonal axes, I, IT, and 711, which deter- 
mine the right spherical triangle shown. The points represent normalized 
test vectors. The test vectors in this figure show a random configuration, 
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which gives no clue as to factorial composition. Hence the cells in the cor- 
responding factor pattern are, in general, all filled. Both positive and nega- 
tive factor loadings are then found in the factor matrix. This is illustrated 


in factor pattern 1. 


Cone configuration 

In Figure 2 we have a cone configuration, in which the normalized test vec- 
tors all lic in a cone with a 45° generating angle. The intercorrelations for a 
cone configuration are all positive or zero. So far, the cone configuration 


Random Configuration 
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Factor pattern 1 


FIGURE 1 


has been of only theoretical interest, and it is included here to bring out 
the fact that, although the correlations are all positive or и the load- 
ings for orthogonal factors cannot all be pues e or zero. s is illus- 
trated in the figure in which the cone configuration DAE be inclosed 
Within the three orthogonal co-ordinate planes. If the rank of the correla- 
lion matrix is 2 or 1 and if the correlations are all positive or zero, then a 

all the loadings are positive or zero. 


factor TOM itten in which 

ctor matrix сап be wri pd à ^ Ge 

When the rank of the correlation matrix 1s 3 or higher, then a positive fac- 
> rank 


tor matrix does not necessarily exist. However, in practice it is usually pos- 
sible to write a positive factor matrix when the correlations are positive or 
zero because the test configuration is rarely, if ever, cone-shaped. 
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Isolated constellations 


In Figure 8 we have a set of three constellations or clusters of test; vec- 
tors. In Figure 3a the clusters are shown at the co-ordinate axes, I, 11, 
and III. When the clusters are separated by acute angles, the clusters are 
correlated, and the co-ordinate axes would then be oblique if they were 
passed through the center of each cluster. The factor matrix 3a shows a 
simple structure with complexity 1 for each test; but, if the factors are cor- 
related, this circumstance is interpreted to mean that the three factors 
themselves have some second-order factor in common. For example, visual 
and auditory acuity might be two factors in terms of which tests may be 
described, but these two factors might themselves be correlated by some 
second-order factor, such as general physical fitness or age. 


FIGURE 2 
Cone configuration 


In Figure 8b we have a different structure and consequently a different 
factor pattern for the same configuration of three clusters. Here axis J is 
placed in the middie of the configuration, and one of the clusters has been 
placed in the plane IIT. The other two clusters are then necessarily one 
positive on III and the other equally negative on III. One cluster is a 
function of only two factors, namely, J and ТТ, while the other clusters in- 
volve all three factors. Factor pattern 3b is not likely 
about the underlying functions as is pattern За. 

The isolated constellations or clusters constitute one of the most easily 
recognized of all the types of simple structure. It can usually be seen by in- 
spection of the correlation matrix when such a configuration is present. The 
tests within a cluster have relatively high intercorrelations, while tests from 
different clusters have much lower intercorrelations. The differentiation of 
constellations was the first type of multiple-factor pattern to be investigat- 


ed, and it is the simplest form of configuration, since each test then has unit 
complexity. 


to reveal so much 


Complete triangular configuration 


A three-dimensional configuration sometimes 


shows the test vectors to 
lie in three distinct planes 


, as shown in Figures 4a and 4b. In thesimple 
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structure and factor pattern of Figure 4a there are both positive and nega- 
tive factor loadings because the configuration extends beyond the ide da 
I-II-III of the three co-ordinate axes. In the simple structure of Pi на 
4b there are only positive factor loadings because the test vectors are a 
fined to the positive right spherical triangle. The test vectors in tke plane 
I-II are described as linear combinations of two factors, and the tira- 
factor loadings are then zero. It will be noticed that all the tests in the tri- 
angular configuration have a complexity of 2, except those which lie at the 
corners of the configuration, which have unit complexity. 


3a 3b 
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` Factor pattern 3a Factor pattern 3b 
FIGURE 3 


Incomplete triangular configuration 
happen in exploratory studies that all the co-ordinate 


configuration. In the three-dimensional case of 
ture in which only two of the co-ordinate 


Planes are defined by the configuration. These are the co-ordinate planes 
I-II and I-III. If the fans of test vectors in the planes I-II and I-III 
extend over right angles and if the dihedral angle at J is a right angle, the 
location of the three orthogonal axes as shown would be fairly convincing, 
even though the plane LI III was not defined by the configuration. In 


It does not usually 
planes are defined by the 
Figure 5 we have a simple struc 
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such a situation one would try to find an interpretation for the factors JI 
and III to determine whether there is some good reason why they should 
not have combined in some of the tests so as also to produce a fan in the 
plane II-III. New tests might be constructed in which these two factors 
are involved and a new factor study made to ascertain whether the inter- 
pretation of factors JI and III can be sustained by the appearance of a fan 
of test vectors in that plane. The present example includes only positive 


[ 1 
I 
Ш I Ш 
4а 4b 
Complete Triangular Configuration 
I п III I п III 
+ + 

sb: 2 
+ T 

= + G + 

T + = d 

ite Р 2x d 

X pan 
+ T + + 
aja + + Р 
^ut E 
+ zb + ate 
i + e + 
m E dra o 
Factor pattern 4a Factor pattern 4b 


FIGURE 4 


factor loadings, but the same general type of reasoning can be extended to 
simple structure with negative factor loadings, as will be shown in other ex- 
amples. 

If the dihedral angle at 7 is not a right angle, the inference is made that 
factors II and III are probably correlated, either positively or negatively, 
depending on their exact locations. If the fans of test vectors from the first 
axis do not extend to a full right angle, as shown in Figure 5b, then the loca- 
tion of the co-ordinate plane [I-III is indeterminate. It might be assumed 
to be orthogonal to J, or it might be placed in the oblique position II'-HI', 
às indicated by the test vectors in the two planes. But such a situation may 
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a the test battery did not happen to contain any test‏ کن اعد 
s hes were functions of I and II near IT, or of I and II I near‏ وا Tn‏ 
байып. т T requent type of ambiguity in the factorial resolution of test‏ 
оа п not profitable to insist that factorial method as such should‏ 
pectin ocations of II and HI in the configuration if these factors are‏ 
r р involved either separately or in some combinations in the battery. I‏ 

is better to make some interpretation of what 77 and III ee д. 


Incomplete Triangular 
Configuration 


Tab RES 


Factor pattern 5 


FIGURE 5 


and then construct new test batteries involving these factors in the purest 
Possible form as well as in combination. If the predicted factorial composi- 
tions are sustained by their location in à defined plane 77-117, the hypothe- 
Sis can be accepted; but, if the new tests scatter in a different manner, the 
hypothesis concerning the nature of II and JI must be discarded. Factorial 
analysis then becomes à scientific method whereby proposed factorial inter- 

Pretations become questions of fact to be determined experimentally. 
It should be noticed that in the interpretation of factorial results we 
Jo not say that it is proved to 


8 Ө К s 7 
Peak of a hypothesis being sustained. We ¢ t 
€ correct. No matter how convincing any experiment may seem to be as 


герат some scientific hypothesis, it is always possible that future experi- 
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ments may show us to be wrong, as the fundamental ideas of a science are 
modified or replaced by more embracing conceptions. What is a discovery 
of interest and importance in one generation becomes inadequate or even 
wrong in the next generation. There is little use in belaboring this limita- 
tion, which is universal for all science. These reservations will be taken for 
granted throughout our discussion of factor analysis as an aid in scientific 
exploration. 
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FIGURE 6 


General and group factors 


Since factor analysis was dominated for a quarter of a century by the cen- 
tral theme of a single general intellective factor with other factors regarded 
as "disturbers" of secondary importance, it is of interest to see perhaps the 
simplest form of general factor in a configuration. One type of configura- 
tion involving a general factor in a battery is shown in the structure of Fig- 
ure6. The simple structure is incomplete in that only two of the co-ordinate 
planes are defined by the configuration, namely, I-II and I-III. Note also 
that all the test vectors have positive projections on the first axis, so that 
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the first column of the factor pattern is filled. All the correlations would be 
significantly positive. The rank of the correlation matrix would be 3. 

The plane II-III is not defined by the configuration in the structure of 
Figure Ga. Some students of factor analysis prefer to locate the reference 
frame for a configuration of this type as shown in Figure 6b. They place 
the first axis centrally in the configuration, so as to maximize the projections 
of the first factor, as shown. The location of orthogonal axes 77 and 111 
can then be made by one of several statistical criteria. The corresponding 
factor pattern is then, in general, filled as shown, the second and third fae- 
tors being both bipolar. The resolution of Figure Gais the simpler and more 
readily interpreted, but the first-factor saturations are not maximized, 

When a configuration is found such as that of the two figures, Ga and 
Gb, there is some uncertainty in the interpretation of all three of the fac- 
tors. The second and third factors may not exist in the absence of the first, 
or they may be correlated with the first factor, in which case the structure 
would be oblique. The first factor is defined by its presence in all tests of 
the battery; but it would be interpreted more clearly if we could predict 
With assurance the tests in which it is absent, as well as the tests in which it 
is present. A new factor experiment should then be made with a variety of 
tests, in which it is predicted that the first factor should be absent but in 
Which one or both of the other two factors should be present. 'The simplest 
and elearest result would be to find a complete triangular structure with a 
group of tests in which the first factor is absent. If the oe үка 
tion is obtained, one has more assurance 1 the cows diens oft антта 
than if the factorial result is left as in Figure б. Whatever the genes а we 
is in a particular battery, its nature is better NESS we gai n m 
With confidence the tests in which it is absent, as yal a sha tas ^ y nan 
it is present, Since factor analysis was for so ү сор өни id on 
cerned with the single general factor and with кокан nie s н бду. 
the procedure here suggested is still avoided io а оки n е. ето 
though it is indicated when the problem is regarded irom the рото 


Of multiple-factor analysis. 
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great variety of configurations in which positive and bipolar factors can be 
combined. 


Dependent composite factor 


Occasionally, but not often, it happens in analyzing experimental data 
that the trait vectors define a secondary plane or hyperplane, as shown at 
II-A in Figure 8 for a three-dimensional configuration. When a secondary 
plane like this appears in the configuration, the interpretation is likely to be 
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FIGURE 7 


of special interest. The test vectors in the secondary plane 77-4 can be de- 
scribed as linear combinations of /, IZ, and IIT, where I and III always en- 
ter into the tests in the same ratio. The configuration of the figure would 
mean that whenever J and JJI combine with IT, then I and IIT enter the 
measurement in some fixed ratio. The test vectors in the three bounding 
co-ordinate planes are all linear combinations of only two of the factors. A 
situation like this should not be ignored when it is found in experimental 
data. It may be suggestive for some fundamental concepts in the domain 
that is being investigated. 
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The factor pattern 

The factor pattern has been defined as a factor matrix in which are re- 
corded the non-vanishing cell entries denoted X and the vanishing entries 
which are left blank. When a differentiation is to be made between positive 
and negative factor loadings, the non-vanishing entries are denoted + or —, 
while the vanishing entries are blank. For the interpretation of factorial re- 
sults the factor pattern is sometimes of greater importance than the numeri- 
cal values of the factor loadings. It is sometimes important to know about 
an unknown factor or function that it is present in certain tests and absent 
in certain others, or that it enters positively (as іп a numerator) in some tests 
and negatively (as in a denominator) in certain other tests. In fact, it is 
More significant to know the factor pattern in this way than it is to know the 
actual numerical values of the factor loadings, because the numerical values 
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in these examples can be generalized to configurations in any number of di- 
mensions. 

Some of the controversy about the general factor is caused by the insist- 
ence of some writers that there must be a general factor, even if it is only an 
average of the test vectors in a battery whose correlations are positive or 
zero. In order to illustrate this situation in three dimensions, we have drawn 
Figure 9, which shows a set of test vectors in a complete triangular configu- 
ration. When these are plotted on a sphere, the location of the reference 
frame, as in Figure 9a, is compelling and convincing. Instead of placing 
the reference frame in relation to the configuration, some writers place the 
first axis at 7, as shown in Figure 9b, because this location is central for the 
whole battery and maximizes the test vector projections. The other axes 


9a 9b 
FIGURE 9 


Alternative locations of first reference axis 


are placed orthogonal to the first central or average axis, and they take posi- 
tions like 7 and ITI outside the configuration so that they necessarily be- 
come bipolar. It is still possible to identify the more fundamental factors, 
A, B, and C, after a central axis has been placed in the configuration be- 
cause, as is to be expected, the first-factor residuals will be largest for the 
tests near the corners of the triangle, and these are the tests 
identify the factors. One type of factorial interpretation th 
mon сап be illustrated in Figure 9b. The third factor wo 
saturations on the tests near B and C, while it would have negative satura- 
tions on tests near A. Such a result is sometimes interpreted to mean that a 
factor such as III in Figure 9b is a “complex relationship" 
If these writers could only be induced to plot their configurat 
at them, there would be less insistence on pl 
the configuration, and the f 


that primarily 
at is rather com- 
uld have positive 


of some kind. 
ions and look 
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chological interpretation instead of appearing as incongruous bipolarities. 
The factors would more often appear as understandable psychological en- 
tities or functions. When a bipolar factor appears in which the positive 
saturations form one or more meaningful groups and the negative satura- 
tions form other meaningful groups, of entirely different character, it is 
quite likely that these groups will resolve themselves in a simple manner by 
locating the reference frame in direct relation to the configuration in the 
manner that has been illustrated in these configurations and factor patterns. 
It must not be expected, however, that a factorial routine can be devised 
which is guaranteed to produce significant results by merely turning a crank 


according to rules. 


CHAPTER X 
ROTATION OF AXES 


Oblique axes 

In previous chapters we have considered the rotation of axes in configura- 
tions that could be completely visualized on a diagram in two dimensions 
or on the surface of a sphere in three dimensions. In actual data the dimen- 
sionality nearly always exceeds three, so that the configuration cannot be 
completely visualized in a single diagram or model. The rotation of axes 
is then determined by two-dimensional sections, one section for each pair 


Table 1 


A Centroid Factor Matriz Fo 


{ IL Ill 
1 B с 
1 — .736 
2 .180 
3 .537 
4 —.209 
5 .182 
6 .519 
li —.452 
8 —.426 
9 .416 
10 2:341 
1 —.147 
12 .485 
13 —.725 
14 .246 
15 .501 
16 257 
17 —.239 
18 —.720 
19 ‚112 
20 .536 


of co-ordinate axes, or by three-dimensional sections of the multidimension- 
al configuration. A four-dimensional configuration is represented by six dia- 
grams when two-dimensional sections are used. A configuration of ten di- 
mensions has forty-five such sections, since this is the number of pairs of 
co-ordinate axes. 

In this chapter we shall use as an example a three-dimensional configura- 
tion with twenty test vectors. This example is used for illustrating several 
methods of rotating the axes, namely, (1) by plotting the normalized vec- 
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tors on a sphere, as shown in a previous chapter; (2) by two-dimensional 
sections, as will be shown in this chapter; and (3) by three-dimensional sec- 
tions in the method of extended vectors, which will be described in chap- 
ler zi. 

We start this problem with a factor matrix Fo in which n=20 and r=3, 
as shown in Table 1. The columns of this matrix represent three arbitrary 
orthogonal axes as determined by the factoring method. These three axes 
are denoted 7, IZ, IIT, and they will be referred to by the subscript m, as in 
previous chapters. These arbitrary orthogonal axes, as well as the test con- 
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direction cosines are shown in the matrix Ao: with elements Xmu, as shown 
in the rectangular notation: 


is Be Oh 
Ifa | 
Wir рЫ 
ш | i] 


Ao with elements Amu 


The subscripts of Ao: show that the columns of this matrix give the direc- 
tion cosines of the initial locations (1) of the unit reference vectors with re- 
gard to the fixed orthogonal frame (0). The subscripts of Xmu refer to the 
axes of the fixed orthogonal frame (m) and the given positions of the ref- 
erence vectors (и). 

For each rotation we denote the given positions of the reference vectors 
Ay, Bı, and C,. By inspection of the configurations in Figures 1, 2, and 3, we 
determine the adjustments to be made in these vectors. Their new posi- 
tions are denoted As, В», and Cs. After the first rotation the plots repre- 
sent oblique axes, so that the true lengths of the vectors are not immedi- 
ately evident. The adjusted vectors are therefore first determined as to 
their direction. They are then called long reference vectors and denoted Аз, 
By, and C}. When they have been reduced to unit length, they constitute 
the new unit reference vectors, As, В», and С». For each adjustment of the 
reference vectors we have, then, two steps in computation. Starting with 
the given unit reference vectors Ai, Bj, Cı, we determine, first, the long 
reference vectors Aj, B5, C4, which give the desired directions. Second, we 
determine the lengths of these vectors, so that they can be reduced to the 
new unit reference vectors As, Ba, Cs. 

The projections of the test vectors on these axes are then computed, and 
a new set of plots is made. By inspection of these plots the next adjust- 
ments are determined by inspection. In the computations for the second 
rotation the given unit reference vectors are denoted Ay, By, Ci, and their 
newly adjusted positions are denoted A», Bs, Ce. In this manner we avoid the 
use of numerically large subscripts, say 10 or 15, for as many rotations. 
T'or each rotation of the reference axes, the given positions are indicated by 
the subscript 1 and the new positions by the subscript 2. 

The given reference vectors will be indicated by the subscript u. The 
new long reference vectors for each rotation will be indicated by the sub- 
script r, and the new unit reference vectors by the subscripts p and q. 

In Figure 1 we have plotted the first two columns, and each point is 
labeled with the number of the corresponding test vector. The points in 
this figure show the configuration of test vector termini as they would ap- 
pear when seen along a line of regard orthogonal to the plane of 4,B;. The 
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two axes A, and B, are in the plane of the diagram. If the axis A, is moved 
in the plane A,B, to the direction of Ag, it will determine a plane (or a hy- 
perplane if 72 3) which will intersect the plane A,B, in the line which is 
marked A-plane. This line is a trace of the plane, or the hyperplane. The 
vectors 7, 13, and 18 will then have vanishing projections on the vector A5. 
In the same manner the given position of the axis B, which is denoted Bı, 
can be moved in the plane A,B, to the direction of B;. Orthogonal to this 
vector is the B-plane, as shown. When the axis B; is so moved, there will 
be a group of test vectors with small or vanishing projections on B;. 

In Table 2 we have summarized the computations for rotating the refer- 
ence axes. Each step in the computations will be described in detail for the 
first rotation, but the computations will be summarized only for the subse- 
quent rotations. The purpose of the rotations is to maximize the number of 
test vectors with zero projections on the reference axes. The given vectors, 
Ai, Bı, and Ch, are represented by the rows of the 3X3 matrix, Si», with 
elements s,,. The three columns represent the new long reference axes, 
here denoted Aj, B5, C}. The vector A; is to be changed to the vector Аз, 
which is orthogonal to the A-plane. The vector A; is a linear combination 
of A, and Bı. It can be expressed in the form 


(1) Al = 1.004, + .90B:, 


as read directly from the diagram of Figure 1. 

In the same figure we determine Bi, which is normal to the trace, 
B-plane, as a linear combination of A; and В. It is read graphically as 
(2) В; = .504; — 1.0051. 


The directions of А; and of B} are so chosen that the majority of the 
twenty points have positive projections. 

The coefficients of these two equations are entered in the columns of 
matrix 512 of Table 2. The columns of that matrix show the direction num- 
bers of the new vectors as linear combinations of the given vectors. Two of 
the columns of that matrix are determined graphically from Figure 1. 

In Figure 2 we have plotted the given columns A, and C; from Table 1. 
In this figure we see the opportunity to move the vector C; to C5, orthogo- 
nal to the C-plane, whose trace passes through a group of points. The new 
vector can be expressed as a linear combination of A; and Cı, namely, 
(3) С; = 40A, — 1.0001, 


as determined graphically, and the coefficients of this equation are entered 
in the third column of the matrix Sis of Table 2. Here, again, the new vec- 


tor C? is drawn in such a direction that the majority of the points have 
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hup 


LoD: = Ат 


Amp 


Table 2 
Rotation 1 


А; By Cr 
Ay 1.00 .50 .40 
В; .90 —1.00 == 
DT s — -1.00 
7 1.90 = .50 = 260 
AL Bi e 
TI 3:00 .50 .40 
II .90 —1.00 -— 
ШЕ = —  -—1.00 
zb| 1.81 1.25 1.16 
VZP | 1:3454 1.1180 1.0770 
0, :7433  .8944  .9285 
1 Вг Cs 
A| .748 лат зт 
B, .669 — .894 — 
Qu. c — — .928 
1.412 — .447 — .557 
Ar B: Cs 
I ‚748 .447 871 
II 669 — 894 = 
ШІ з= — —.928 
Z| 1.412 — .447 — .557 
EH В» Cr 
As | 1.000 
B, |— .206 ‚999 
B .276 .166 .999 
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positive projections on Су. Figure 3 was plotted from columns B and C of 
Table 1, but this figure was not used in this rotation of the reference frame. 

The first step in the rotation of the reference axes is to determine graphi- 
cally the numerical entries in the matrix Si» with elements su. The sub- 
scripts of Syo indicate that this matrix shows the adjustments in the ref- 
erence vectors from position 1 to position 2. The elements Sur represent the 
relation between the given unit reference vectors, и, and the new long ref- 
erence vectors, r. 

In order to determine the true lengths of the long reference vectors As, 
Bi, Сї, they should be referred to the given orthogonal reference frame.- 


С 


FIGURE 2 


‘This frame is actually represented in A1, Bi, and C, of the first three fig- 
ures, but it is not so directly represented in the diagrams for subsequent ro- 
tations. Let a matrix Los, with elements lmr, represent the long reference 
vectors r in terms of the fixed orthogonal frame m. The subscripts of Los 
indicate that the long reference vectors in the second position are expressed 
in terms of the fixed orthogonal frame (subscript 0). | 

The columns of the matrix Ao: with elements Amu show the direction со- 
sines of the reference vectors in their positions 1. Since the initial positions 
of the reference vectors are in the fixed orthogonal frame, we have A, in 
the axis Т, B, in the axis II, and C, in the axis III, so that Ani is an identity 
matrix. The elements of the matrix A are denoted Amu when it represents a 
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set of reference vectors before adjustment. The elements of A are denoted 
Amp When it represents the reference vectors after an adjustment has been 
made. 

Let Ao» be a matrix with elements Amp whose columns show the direction 
cosines of the reference vectors in their second positions. The columns of 
this matrix show the direction cosines of the unit reference vectors p in the 
second position, expressed in terms of the fixed orthogonal frame m. 


FIGURE 3 


In order to determine the true lengths of the long 


reference vectors A5, 
Вз, and Сз, we describe them in terms of the fixe 


d orthogonal frame. We 
then have 
=" 
(4) D Nus = Be 
u 
which can be written in the rectangular form 
(5) 
da BE G AS B: e = Af B; с 
I} 1 | En 1.00 | .50 .40 I 1.00 50 .40 
п 1 Bı .90 | -1.00] — H| 001-1 001 — 
и | Tel sey | | -1.00| п EE 
| 
Amu s 


Sur lmr 


ROTATION OF AXES 201 


M ke ae 
| а j , 1,111. Inthe matrix Ssu we have the 
new vectors 44, Bi, and С; expressed in terms of the given vectors Ai, B 
and e. The matrix multiplication gives the product matrix with Meme, 
lr, in which the long reference vectors Aj, By, and C are also expressed n 
terms of the fixed orthogonal unit reference vectors. 'The matrix X i is a 
unit matrix for the first rotation. Consequently, Sur and lmr are here iden- 
tical, but they will differ for subsequent rotations. 
The matrix equation can also be written in the form 


(6) Ло == Los, 


where the columns of Los show the direction numbers of the long reference 


vectors in position 2 in terms of the fixed orthogonal frame. 

Since the columns of Loz in Table 2 show the projections of the long vec- 
tors on the fixed orthogonal reference frame, we can determine the true 
lengths of the vectors 44, B4, and C}. The sum of the squares in each col- 


umn is written 


2 
2 "nr! 
— 


m 


or, more briefly, as XP on the data sheets. The square root of each sum is 
tor r. The reciprocal is 


УР, and it is the length of the vec 


(7) d. E 
rp 
30 


Jements of a diagonal matrix D». 


arded as the e 
the long reference vectors 


tations is to reduce 
the relation 


| һе values of d,, may be reg 
t The next step in the compu 
© unit length. This is done by 


(8) УФ» = Ap: 


3 


Or, i Н 
› In matrix form, 


(9) LoDs = Aor» 


= 
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which can be written in rectangular form: 


(10) 
Lo D. T 
AL в oC As Be Eoo das A e 
1| 1.00] so)  .40| az | таз D] auz| 371 
IL .90| —1.00 — B: . 894 Il 669 | — . 894 — 
TOs) hess Ml ses zm e 99| ш| —| — [оов 
la dry Amp 


The oblique factor matrix which is the result of this rotation can now be 
computed by the equation 


(11) > аһАтр = Vjp 
эп 
or, in matrix form, 


(12) FoAos = Ve, 


where P, is the given factor matrix and Vs is the oblique factor matrix. This 
is the factor matrix in its second position. If the rotation of reference axes 
were limited to orthogonal transformations, we should have 


(13) PoAo2 = Fa " 


where Fa would show the projections of the test vectors on a new set of 
orthogonal axes. 

If it is desired to compute each factor matrix V from the just preceding 
factor matrix, instead of making the computations at each rotation from 
the original orthogonal factor matrix Fo, then we use the transformation 
matrix hyp instead of №. The columns of the matrix H with elements hup 
show the new unit reference vectors А», В», and Cs in terms of the preceding 
reference vectors А, Bı, and С. We then have 


5 


(14) Sis 


Surdi = hup 
= 


or, in matrix form, 


(15) SisDs = His 
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or, in rectangular form, 


(16) 
Se D: Thy 
E B: C: 4. B: C: = Ae B: Ca 
Ay 1.00 ‚50 40| A: | 743 Ay 1.743 447 ‚371 
В, .90| 21.00] — B; .894 В, |.669| —.894) — 
Ci = = —1.00| C: JW Cr | = = [~= 
Sur dep hup 


The oblique factor matrix V» can be determined from the preceding 
factor matrix V; by the transformation Мз. We then have 


(17) Vilis = Vs. 


The rotational problem starts with the factor matrix Fo with elements 
аы, Which give projections of the test vectors j on the fixed orthogonal 
frame m. The first step is merely to assign the initial locations of the trial 
reference vectors in the orthogonal axes of the fixed frame. This is repre- 
sented by the transformation 


Foo = Vi, 


in which Ао is an identity matrix. The first rotation represents the inde- 
pendent adjustments of the reference vectors from their initial positions in 
the fixed axes. This adjustment is represented by the equation 

Fokos = Ve, 


Ao» give the direction cosines of the new reference 


in which the columns of n à 
] vectors т. The axes of Vs are then, 


vectors in terms of the fixed orthogona 


in general, oblique. 


The second rotation is represented by the equation 


FoAos = Ёз, 


where the columns of Ao; show the direction cosines of the adjusted refer- 
ence vectors. The oblique factor matrix Уз can be obtained from the next 
preceding factor matrix, Vs, if desired. The relation then is 


ҮН» = Vs. 
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The cosines of the angles between the new reference axes should be de- 
termined, to make sure that no pair of them is approaching another pair so 
closely as to represent practically the same factor. If that should happen, 
the dimensionality of the oblique factor matrix would be reduced so that 
the columns would be linearly dependent. It would then be necessary to alter 
the location of one or more of the reference axes. With a little practice this 
error can be avoided. The cosines of the angles between the reference vec- 
tors are determined by the cross products 


2 Са 


(18) 25 we 


m 


or, in matrix form, 

(19) ЛАА = С. 
After the first rotation we have 

(20) ANo = C... 


It is useful to record these cosines on the corresponding diagrams for the 
next rotation, so as to avoid moving any pair of reference vectors so close 
together that a dimension is lost. The diagonal values of A’A must be unity, 
since unit vectors are involved. 


The successive steps in the computations for the first rotation are shown 
in Table 2, and they can be summarized as follows: 


1) Write the elements of the matrix Siz by inspection of the plots for 
Vi. 


2) Perform the matrix multiplication 
(6) Ао = Los, 


where Ao; is a unit matrix for the first rotation. 


З) Determine the stretching factors d,p from the sums of squares in col- 
umns of Los. 


4) Compute elements Anp of As» by the equation 
(9) LoD: = Ap. 


5) If it is desired to compute the oblique factor matrix, Vs, from the next 
preceding factor matrix, Vi, then compute the elements hup in the matrix 
His by the equation 


(15) abs = His. 


ROTATION OF AXES 205 


6) Determine the cosines of the angular separations of the new unit refer- 
ence vectors by the equation 


(20) Абы = Cr, 


7) If these cosines are acceptable, compute the new oblique factor ma- 
trix, Vs, by either of the two following equations: 
a) In terms of the original factor matrix, we have 


(12) Роло = Vs. 
b) In terms of the preceding factor matrix, we have 
(17) Vitis ج‎ Vo. 


In the first rotation the initial factor matrix, Vi, is identical with Fo, which 
is the initial factor matrix obtained from the experimentally determined 
correlation matrix. The matrix Va is shown in Table 3. 

8) Plot a new set of diagrams for all pairs of columns of V» and examine 
them to determine the adjustments, which will be the next rotation of 
axes. 

It may be helpful to visualize in a single table the computations that are 
summarized in Tables 2-7, inclusive, for the several rotations of the refer- 
ence axes in the present illustrative problem. The formulae have been ar- 
ranged in order of computation and in a separate column for each rotation. 
The subscripts indicate the rotations to which the formulae belong. The 


same schema can be extended, of course, to any number of rotations. 


Computational formulae for three rotations 
Given the factor matrix Fo= V; as obtained from the correlations and 


the identity matrix Ло. 


Rotation 1 Rotation 2 Rotation $ 
Determine S graphically Determine S»; graphically Determine S34 graphically 
До = Lo: Ao2S23 = Los Aosa = Los 
D, from Lo D; from Los D, from Los 
[S23D; = Hes] [Sa4D4 = Hai] 
LoD: = Ao LoD: = Ам 
АозАоз = Cs Ado = OG 
(Нз = V; (аза = Fy 
{or , or 
(Fotos = Vs Lia sc T 
(In first rotation these two (These are two alternative (These are two alternative 
formulae are identical) formulae) formulae) 


Plot pairs of columns of V Plot pairs of columns of Vs Plot pairs of columns of V, 
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It should be noted that the matrix product of the several transformations 
H is equal to the transformation A for the same rotations. We then have 


(21) HoH 2H 3H зд = Мол, 


where the transformation Но = Ао =I. The transformation hup carries a fac- 
tor matrix V,_; to the next factor matrix V,, while the transformation Ao, 
carries the original factor matrix Fo to the oblique factor matrix V,. In gen- 
eral, the computational labor is least when the oblique factor matrix is com- 
puted by the transformation H; but if a matrix-multiplying machine is avail- 
able, the computations are performed best by the transformation A. 


COS--266 


FIGURE 4 


The result of the first rotation of axes is shown in the next three figures 
(4-6), where pairs of columns of V; are plotted as before. It should be noted 
that, while the reference axes are now oblique, they are nevertheless plotted on 
orthogonal co-ordinate cross-section paper, as indicated in the figures. The 
procedures here described do not require that the plots be actually made 
with oblique axes on the graphs. 

In stating each new rotation we denote the previous axes with the sub- 
script 1 and the new axes with the subscript 2. In Figure 4 we have plotted 
the columns A and В of Table 3. The vector B; is moved to the position B4, 
so that the trace of the B-plane goes through a group of points 8, 15, 20, 6, 
9, 12. The new vector B; is described from the graph by the vector equation 


(22) В, = 1.008, + .104,. 


ROTATION OF AXES 207 


In Figure 5 we have plotted the columns АА and C of Table 8. A radial 
trace for a new A-plane can be drawn, as shown, with the normal Aj. Such 
a move of the A-axis will not only retain the zero, or near-zero, projections 
of 1, 18, and 18, which are at the origin, but will add six more tests to the 
list of negligible or zero projections on the A-axis. The new reference axis 
А; сап be expressed as a linear combination of the axes A, and C, of Figure 5. 
This relation is 
(23) А5 = 1.004; — .720,. 


In the same diagram we see the opportunity to move the C-plane so as to 
obtain zero projections not only for those tests which are shown at the 
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COS = #276 
FIGURE 5 

origin but also for the group 8, 5, 15, 20. The normal to the C-trace is C4, 
which can be written in the form 
(24) 5 = 1.00€; + .254;. 

In Figure 6 the trace of the C-plane is drawn radially so as to include the 
group of tests 1, 13, 18, and its normal is C5, which is written 
(25) "9 = 1.000, = .12B;. 
In the same diagram the trace of the B-plane is drawn radially so as to pass 
through the group 2, 14, 19, and the normal is written 


(26) B} = 1.008, — .20C; . 
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The numerical values in equations (13)-(15) are entered in the Ss5-matrix 
of Table 4. Here it will be noted that a new process has been introduced, 
namely, the combination of movements of a plane from more than one dia- 
gram. The new vector B, is determined here as a sum of three vectors, 
namely, the previous B vector plus two corrections, namely, +.104,, and 
—.20C;. In a similar manner, the new vector C; is determined by two cor- 
rections added to the previous vector C, namely, 4-.254; and —.12B,, as 
shown in the columns of 5»; in Table 4. 

In combining several corrections in the same rotation it is useful to re- 
member that the corrections should not be large—say, not over .30. If a 


COS = +166 
FIGURE 6 


rotation whose sine is greater than about .30 is indicated on a diagram, it 
should be taken singly for that rotation. If several small rotations are indi- 
cated on the diagrams, they may be combined if they are independent, The 
independence of two small rotations can be seen in the present examples. 
The new vector C; is determined by two corrections from Figures 5 and 6. 
In Figure 5 the trace of C is moved toward the group 3, 5, 15, 20, and these 
are near the origin in Figure 6. In Figure 6 the trace of C is moved toward 
the group 1, 12, 18, and these are near the origin in Figure 5. Hence the two 
movements can be combined. If a trace can be moved toward a certain 
group of tests in several diagrams, the move is made on only one diagram, 
for otherwise there would be an overcorrection. Another useful principle is 
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Table 4 


Rotation 2 


Table 5 


Factor Matrix Fotos = Уз 


At nt с: Az В: Cs 
A, | 1.00 .10 .25 1 | — .082 .970 001 
В, 1.00 — .12 2 .026 055 .938 
GS Ies E = 20 1.00 3 .954 — .058 .016 
4| — .021 .500 723 
28 .90 1:18 5 811 .330 .002 
6 .800 — .031 887 
7| — .071 737 .510 
8 .814 825 O11 
7 9 .462 — .017 756 
al в! c 10 | — .043 636 621 
= = 11 .582 .626 .002 
I „476 Em .503 12 .642 — .030 .605 
Il .669 — .829 2275 13 | — .090 .958 .032 
I . 66S .186 — .928 14 .088 .001 .919 
= — > 15 .931 — .037 .026 
x| 1.813 — .195 — .151 16 .688 .246 . 356 
17 .231 .641 .456 
se | 1.12004 .9216 1.1898 — 18 | — .070 .946 .035 
SE 585 — .9600 1.0908 19| — .024 .104 .905 
VP LT 1.0417 9168 20 .945 — .060 .026 
HE f 
Ch| 7.077 7,848 7.208 
$| 7.073 7.34 7.901 
As В: С: Р E Ux xi 
1 .945 .104 .229 
р 1:042 — .110 
Ci |— .680 — .208 -917 
z 264 .938 1.036 
Ws B: C: 
450 466 -461 
111 — 1864 -252 
III ‚631 ‚194 — .851 
s| 1.7138 — +209 — .188 
Ae Be E: 
E 1.000 
es 4 1.001 
fal =e 1.000 
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that if a trace moves toward the configuration in one diagram (as C in 
Figure 6) and away from the configuration in another diagram (as C in 
Figure 5), then the two corrections can be added in the same rotation with- 
out checking the numbers of the variables. 

Another example of the combination of two corrections is the B-trac' 
in Figures 4 and 6. Here, again, one of the movements is toward the con- 
figuration (Figure 6) and the other is away from the configuration (Figure 4). 
'The two movements are written as follows: 


(27) B} = 1.008, — .20С\, 
and 
(28) B; = 1.008; + .104,. 


The two corrections are shown in the S23-matrix of Table 4. 
The steps in the computations of Table 4 are as follows: 


1) Compute the elements of Los by the relation 
Аоз$оз = Los. 


2) Compute the diagonal elements d,p of the diagonal matrix Ds. 
3) Compute the elements of the matrix Aos by the relation 


LoD: = Aos. 

4) Compute the cosines с, of Сз by the relation 
Aostos = Cs. 

5) Compute the elements of the next factor matrix by the relation 
Foos = V3. 


6) Plot pairs of columns of V; on orthogonal cross-section paper and re- 
cord values of с, on each diagram. 


The computation of H»; is optional as an alternative method of arriving 


at Vs. If it is desired to compute V; from Vo, then the computation is made 
by the formula 


ҮН» = Үз E 
If it is desired to compute V; from the original factor matrix Fo, then we use 
the formula 

Foos ы; V3 А 


In any event, the latter formula should be used for check on computations 
after five or six rotations have been made by the transformation И. 

The three plots for Уз of Table 5 are shown in Figures 7, 8, and 9. These 
plots show that the reference axes are nearly in their final positions in the 


Figure 7 


Ficure 8 
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configuration. The A-plane is adjusted slightly in Figure 7. The B-plane 
is given two corrections, as shown in Figures 7 and 9. The C-plane needs no 
further adjustment. These adjustments are shown in the matrix S,, of 
Table 6. The computations of Table 6 proceed in the same way as for the 
previous rotations. The oblique factor matrix V; is computed as shown in 
Table 7. 

When the factor matrix У; of Table 7 has been computed, the three pairs 
of columns are plotted as shown in Figures 10, 11, and 12. Inspection of 


Ficure 9 


these three plots reveals the simple structure which became evident even 
after the first rotation. Since no further rotations are indicated in Figures 
10-12, they are accepted as the configuration for interpretation. This phase 
of factor analysis cannot be illustrated in the present problem, which is 
fictitious. The reference vectors are denoted A,, as shown in the columns 
of the oblique factor matrix V4. The cosines between these reference vec- 
tors are shown in the matrix C4 of Table 6. It is seen that these cosines are 
negative, from which we infer that the angles separating the reference 
vectors are obtuse. Hence the angular separations of the corresponding 
primary factors Т, can be expected to be acute. The interpretation is that 
the primary factors are positively correlated. 


Table 6 


Rotation 3 


Table * 


Factor Matriz FoAo = Vs 


Matrix Хи T в; с: m Ав Ac 
A, | 1.00 .05 1 .006 965 001 
T ^00 1.00 2 .032 .000 938 
i 6 — .05 1.00 3 :964 .010 .016 
vel 4 .025 .462 .723 
1.09 1.00 1.00 5 .854 .370 .002 
6 .810 .009 .387 
т | — .003 707 510 
8 .395 .840 .011 
9 .468 .031 .156 
бөз 11 B ct 10 .015 .602 .621 
11 649 654 .002 
I .492 .465 16 12 .649 .027 -605 
П 1554 == .845 252 13 | — .003 :951 .032 
T ИП .648 268 — .851 14 .090 .040 .919 
= 15 .943 .012 .026 
1.694 — .112 — .138 16 .721 .263 .356 
17 .294 .629 .456 
se| .9689 1.0021 1.0002 18 .016 943 .035 
/SB| 9843 1.0010 1.0001 19) — .014 .058 .905 
Ж v t 170160 9990 9999 20 .955 .013 .026 
Ch | 7.865 7.3355 7.203 
= =| 7.866 7.335 7.201 
SuDi=May As B: C: 
Aı | 1.016 .050 
By ‚091 .999 
50 C — .050 1.000 
р 1 
1.107 .999 1.000 
[ТИЛЕК As В: e 
500 .465 461 
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The correlations between the primary factors T, can be determined by 
the relation 
(29) Ry = D(A) ~D, 


where the diagonal matrix D is so chosen that the diagonal elements of Rp, 
are unity. The result is shown in Table 8. The procedures here described 


Table 8 
Correlations between Primaries T, 


= 
COS=- 226 


FIGURE 12 


are applicable for rotation of axes in any number of dimensions. The in- 
vestigator must make his own decision as to whether the axes are to be re- 
stricted by being orthogonal or are to be allowed to follow the configuration, 
asin the present example. He must also make his own decision as to whether 
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to restrict the factors to the positive manifold. It is probably best to impose 
none of these restrictions and to allow the reference vectors to be oblique 
if the configuration demands it or to allow each factor to be either positive or 
bipolar. 

There are several ways in which the results of a rotation can be antici- 
pated. These become apparent with practice. Several have been described 
here. It is useful to be able to predict whether a rotation will increase or 
decrease the cosine of the angle between two reference axes. This can be 
illustrated by Figure 4. The cosine of the angle between the given vectors 
A4 and Ав, denoted A; and B,, is —.266. The negative sign means that they 
are separated by an obtuse angle. If the vectors should be orthogonal, then 
it is clear that A; must move toward Ву, or else B; must move toward А |. 
In this diagram, B, is moved toward A, to the position B4. Hence we can 
expect that the cosine will be increased toward zero unless the vector A, is 
also moved cither by this diagram ог by some other diagram. If the cosine 
of the angle between two reference vectors were positive, then they would 
have to separate in order to make them more nearly orthogonal. When we 
are dealing with cosines of large absolute value, these rules are uscful in 
avoiding a rotation by which two reference vectors might be made nearly 
collinear, in which case one of the dimensions would be lost. Special cases 
sometimes appear in large studies in which these principles are adapted in 
different ways, but these become evident with practice. It is unfortunate 
that the rotational problem cannot be resolved in some routine which can be 
applied without judgment. The reason is that in dealing with actual fallible 
data we frequently meet with an incomplete simple structure which is over- 
determined for some hyperplanes and not for others. Furthermore, we deal 
with some positive or bipolar factors that are clear and overdetermined, 
some factors that are only suggested by an almost indeterminate configura- 
tion in some dimensions, and with the unique variance and error variance, 
which obscure the less clearly defined hyperplanes. These conditions are de- 
termined by the test batteries that are assembled for investigation and 
which call for new factorial experiments. Factorial method as such does 
not give a determinate solution for an ill-conceived factorial experiment. 
This is, in one sense, fortunate because it becomes a question of experimen- 
tal fact whether a hypothesis represented in the test battery is sustained in 
the factorial analysis. A principal-axis solution is always available, and it is 
unique for any arbitrary test battery; but it does not, except in special cases, 
enable us to ascertain whether a psychological hypothesis about factorial 
composition is sustained or refuted. 


A single-plane method of rotation 


The rotation problem is, for most computers, one of the most difficult in 
factor analysis. The reason is in the cireumstance that most of the rotational 
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methods require some experience before the graphical representations can 
be handled effectively. It is our purpose in devising the present method to 
provide a procedure which can be handled by a clerk for most of the work. 
The method will be referred to as a single-plane method of rotation. The 
method seems to require little more experience than is necessary to estimate 
the slope of a line through the origin for a set of points. Questions of inde- 
pendence of rotation are minimized in importance, and the method seems 
to be convergent, so that a hyperplane can be located without a discourag- 
ing amount of labor. The computer need not think about the summation of 
vectors or about the distinction between a reference vector and its trace in 
the diagram. The method does not postulate a positive manifold, so that 
it can be used as well for locating positive and bipolar factors. The method 
proceeds by locating one hyperplane at a time, so that the investigator can 
begin the more interesting job of interpretation of each factor while the 
remaining ones are being computed. This method has only recently been 
devised, and it has been tried on several sets of experimental data with 
promising results. 

While it is not necessary for a clerk-computer to understand the theory 
of the method, it will be given first and then the routine which can be ex- 
plained to a novice in computing. 

In Figure 13 let Q represent a trial unit reference vector whose direction 
cosines are denoted М. The fixed orthogonal reference axes are denoted 
by the subscript m, as in previous factorial work. We shall denote the given 
trial reference vector Q and the adjusted trial reference vector P. This will 
be done for each trial, so that the reference vector P that is obtained in one 
en trial vector Q for the next adjustment. This process 
continues for, perhaps, five or six adjustments, until the reference vector 
seems to be located. If there is no simple structure or if the particular hy- 
perplane is not defined by the configuration, that fact becomes evident. 

It is first assumed in Figure 13 that the trial vector Q is orthogonal to 
one of the orthogonal reference axes Mas shown, but we shall see later that 
this restriction can be overcome. The projections of the test vectors J on 
the trial vector Q are computed and denoted vj, (not shown on diagrams). 


We then have 


(30) би = D, nes 


т 


trial becomes the giv 


where aj», are the direction numbers of the test vector J with respect to the 
jm К 


fixed orth 1 frame M. i 
Шо wees of the unit vectors of the fixed orthogonal frame, we al- 


ready have the projections of the tests on this vector. They are the values 
а, in a column of the given factor matrix F. lf M in the figure represents 
the first centroid axis, then the projections (M3) are given in the first 


FI янпо] 
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Ae 


° 
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column of F. In the figure we have represented, by a set of points, the 
paired values from a column of F and the column of vj,. The object is to 
draw a line through the origin with a slope so chosen that a large number of 
the points will lie close to the line. This has been illustrated in Figure 18, 
and the slope s; is noted. It is simply the ordinate on the line TT for 
а= +1. 

Let L be a vector so defined that (1) it has unit projection on Q and (2) 
it is orthogonal to the line ТТ. The vector L can be expressed 


(31) L2Q-—sM. 
Let lma be the direction numbers of L. Then we have 
(32) lma = Ame — 8, 


since the vector M is a unit vector on the co-ordinate axis M. The new 
vector A, will be collinear with L and of unit length. 

Let us now consider the oblique case, which is illustrated in Figure 14. 
The trial vector Q has the projection X; on the vector M, as shown in the 
figure. The projection Аһ then is one of the direction cosines of Q, and M 
is in one of the fixed orthogonal axes. When the values of vj, and ajm are 
plotted on orthogonal cross-section paper, the diagram will look like Fig- 
ure 13. The vector (—8M) is shown in Figure 14, where it is, of course, 
parallel to ЛГ. Here we write 
(33) U=Q-sM, 
where U satisfies only one of the requirements for L. The vector L is or- 
thogonal to ТТ, and so is the vector U. But the vector L is also defined as 
having unit projection on Q. The vector U can be extended by a stretch- 
ing factor k, so that it has unit projection on Q. Then we have 


(34) L-RU, 


as shown in Figure 14. In order to determine the stretching factor for U to 


give it unit projection on Q, we write 
(35) k(UQ) = 1, 
80 that 

ее 
(36) = 0070) ' 


where UQ is a scalar product. From (33) and (36) we have 


1 
(37) k= O — «MQ 
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© k = 1 - 
(38) b 0 0: 


and, since Q is a unit vector, we have 


І emm 
(39) dT н 
But the scalar product MQ is merely the direction cosine Аһ, for the axis M. 
Hence we write 


1 
(40) k= PEE 
The direction number of Q for the axis M is Ama, as shown in the figure. 


Since the vector (—sM) is parallel to M, the direction number of the vector 
U for the axis M is 


(41) Ung = № = 8. 


The direction number of the vector L for the same axis is Ing, and it is 


(42) 


Ing = Kung, 


so that the desired direction number can be written in the form 


№ — 8 
43 ЕТ rd 
(43) Ing 1 = Sng ` 


which is a simple computing formula. Here the Ang S are the direction cosines 
of the given unit trial vector Q. The slope s is read directly from a graph. 
If there are r columns of the factor matrix F, then there are as many graphs 
to be plotted. Each one gives a slope value sm, which is set equal to zero if 
no rotation is indicated. 

When formula (43) has been evaluated for each of the orthogonal axes 
M, the new vector L is determined. It is then normalized and becomes the 
new trial vector P. 


A numerical example 


A three-dimensional example will be used to illustrate numerically this 
method of rotation. Table 9 shows a factor matrix F for twenty variables 
and three orthogonal factors. This table is the same as part of Table 3 of 
chapter vii, except that it is here recorded to only two decimals. The prob- 
lem is to locate the reference axes A», Which define a simple structure. 
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The procedure starts with the selection of a test vector. In the present 
example, test 15 was arbitrarily chosen as a starting vector. Its direction 
numbers are shown in row 15 of F in Table 9. These three direction num- 
bers, аы, are recorded in the first column of Table 10, trial 1. This vector 
is normalized, and it then becomes the first unit vector A, with direction 
cosines Ау, as shown in the second column. The computations for nor- 
malizing the given test vector are shown, in part, in Table 10. We then have 


Table 9 
Factor Matrix F Plane A 
Trials 
I п ш 
° 3 
1 66 —.74 14 — .04 .01 
2 72 18 —.66 — .03 ‚02 
3 66 ‚54 50 .94 .96 
4 $87 -21 —.44 = M -03 
5 83 18 51 .82 .85 
6 84 .52 15 77 .81 
7 86 —-.45 —.27 = .07 .00 
8 85 —.43 32 84 .40 
9 ‘86 .42  —.30 42 41 
10 "gg —.34 —.35 = .05 .02 
11 189  —.15 dd .60 .65 
12 .88 .48  —.09 .60 .65 
13 67 —.72 m — .05 .00 
14 72 95 —.62 .04 .09 
15 63 ‚50 .52 .92 .94 
16 94 .26 16 67 72 
17 97 —.24 —.08 23 .30 
18 б —.72 mi = .03 „02 
19 `70 Alo —.65 = 07 — .02 
20 `66 ‚54 .49 .94 96 
Е 15.71 —.02 05 6.93 7.88 
Table 10 
pens x ; des 0 IA l Amp Sas 9173 c-1044 
ajm mq 
n 4 .49 .44 c= 90827 
Trial 25 {1 16 .84 4 = sos 
al 1 i E 00 718 .70 —.09 1.09 .61 58 
HI "52 154 —.85 .89 —.19 1.19 S5 . 
жазысы 
nicus 5 1—08 1.0% AY -50 с=1.0179 
Trial 2 I P e 2 .03  .97 .55 -56 9936 
i 88 05 163 208 197 .65 .6 a 


222 MULTIPLE-FACTOR ANALYSIS 


Ya? = .9173, as shown, and 1 Мх e, which is the multiplying factor by 
which 


Са» = Мы, 


where с= 1.0441. 

The projections ® are then computed and recorded as shown under trial 1 
in Table 9. The column vj, for the first trial is plotted against each of the 
three columns of the factor matrix P, and the three resulting diagrams are 
shown in the top row of Figure 12. On each diagram a line is drawn through 
the origin so that it goes through groups of points wherever possible. Large 
angular deviations of 30°-45° from the horizontal are not made unless the 
configuration of points clearly demands so large an angular deviation from 
the X-axis. The slope s; of each line is noted, and it is recorded in the next 
column of Table 10. The slope of each line is read graphically. It is read 
directly from a vertical line through the point +1 on the base line. The 
rest of the calculations for trial 1 are self-explanatory, as shown in Table 10. 
The column Ing is normalized and then becomes the column Anp. The cal- 
culations show Z/— 1.2122, which gives the multiplying factor c= .90827. 
The column Amp shows the direction cosines of Ap, which is the trial vector 
A, for the next trial. 

When the direction cosines Amp Of the next trial vector have been deter- 
mined, it is useful to compute the scalar product of the given vector Q and 
the new unit vector P. This is the cosine of the angular displacement ¢, 
represented by the trial in question. The corresponding value of sine ¢ is 
recorded as shown. For the first trial, sin ¢=.28. It will be found that 
trials should be continued until sin ¢ is about .10 or less. In the present 
example it is found that the second trial gives sin ф= 11, which is small 
enough that two adjustments are considered sufficient in this case. 

In the second section of Table 10 these direction cosines are copied in 
the column Amg, as shown. The projections v;, are computed and recorded 
in Table 9 in the column for trial 2. This column is plotted against the 
columns of the factor matrix F, and we then have the three diagrams in the 
bottom row of Figure 15. Here the groups of points are closer to the hori- 
zontal line in all diagrams, a fact which shows that the desired solution is 
being approached. A line through the origin is drawn on each of these 
three diagrams which goes through the concentrations of points. The slopes 
5» are recorded in Table 10, and the rest of the computations proceed as 
before. 

The value of sin ¢=.11, which is small enough so that further rotation 
is not indicated. The projections v;, of the test vectors on the reference vec- 
tor determined in two trials are computed and recorded in the last column 


of Table 9. "This last column is, in fact, one column of the oblique factor 
matrix V. 


GI HAI] 


ооо о. 
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In locating the next reference vector we start with one of the test vectors 
which lie in or near the plane already determined. We choose, therefore, 
one of the vectors that have small values in the last column of T'able 9. We 
might start with any one of the vectors 1, 2, 4, 7, 10, 13, 14, 18, 19. These 
would not all lead to the same plane, but it does not matter in which order 
the planes are determined. The starting vector for determining the third 
plane is chosen from those test vectors which have small projections on 
both the first and the second reference vectors Ау. In this manner one avoids 
starting with a vector that leads to a plane that has already been deter- 
mined. If the configuration is indeterminate, this fact becomes evident 
when there is no clear indication of where to draw the lines on the graphs, 
or when the adjustments indicate large deviations that would carry the 
new trial vector into one of the reference vectors that have already been 
determined. 

It has been found that inexperienced student assistance can be used on 
these computations, most of which can be arranged with check columns. 
The check sums have not been shown here for Table 10. It is a fortunate 
circumstance that an error in these calculations is self-correcting, in that 
the adjustment is made by the computations in the next trial. It is best to 
provide rather complete check columns so that each step is self-checking. 

The work of plotting the diagrams can be considerably reduced by not 
plotting the points for which a is smaller than v. These points have rarely 
any effect in determining the slope of the desired line through the origin, 
and they might as well be omitted in plotting the diagrams. 

When all the planes have been located by this method, either as planes 
of considerable variance or as residual planes, the columns of the oblique 
factor matrix V should be plotted in pairs. Small final adjustments can 
then be made, if necessary, by the methods described earlier in this chapter. 

The present method has been tried on several large factor studies, on 
small test batteries, and on fictitious examples, and it has so far been found 
to be rapidly convergent to the solution. One of its advantages seems to be 
that the computer need not worry about the independence of the several 
angular adjustments in each trial, because each of them affects only one of 
the direction cosines of the new trial vector. The difficulty caused by over- 
correction does not seem to be a problem with this method. 


CHAPTER XI 
THE METHOD OF EXTENDED VECTORS 


This method has proved to be effective in analyzing a factorial matrix of 
more than three dimensions, but it is applicable only in those cases in which 
a central reference vector can be found on which all the test vectors have 
positive projections. The method of extended vectors will be described, first, 
for a simple ease of three dimensions, in which the new method gives the 
solution in one diagram if a simple-structure solution exists. The method 
will then be described with a five-dimensional fictitious example, which il- 
lustrates, perhaps better than the simpler example, just how the method is 
used in practice. 

The method of extended vectors for three dimensions 
apter we described the method of rotation of axes by 
two-dimensional sections, which are represented on diagrams. Each dia- 


gram represented the plane determined by two of the reference vectors, and 
it was plotted from pairs of columns of the factor matrix V. In the method 


of extended vectors these sections represent three dimensions. The prin- 
ciple of the method will be explained briefly before the numerical example 


In the previous ch 


is given. х . As 
Let Figure 1 represent a configuration of test vectors with origin at О. 
Let the axis Z represent the first centroid axis, the major principal axis, or 
any other axis that is central in the configuration and on which all the test 
vectors have positive projections. Let the are CC represent a ш врһеге 
With center at О, and let the reference vector 7 be of unit length. Consider, 
next, the plane PP, which is tangent to the unit sphere and orthogonal to 
the central axis Т. In a configuration of r dimensions this plane becomes an 
(r — 1)-dimensional hyperplane. Let each test vector be extended so that 
its terminus is in the plane PP. All the test vectors will then have unit 
Projection on the axis Т. Each test vector is extended by the multiplier 


(1) n 


axis is unity. This has been done in Table 1. 


The extended faetor loadings were obtained from Table 1 of the previous 
Chapter. The multiplier De for test j is applied to ап шенбе loadings ау 
of that test to determine the corresponding extended factor loadings A jm of 
Table 1. 


So that its projection on the I- 
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In Figure 1 let OAB represent one of the co-ordinate planes containing a 
number of the test vectors. When these have been extended, their termini 
also lie in the tangent plane PP. Since the intersection of two planes is a 
straight line, it is clear that the termini of the test vectors in the co-ordinate 
plane OAB will lie in a straight line in the plane PP and that this linear ar- 
rangement of test vector termini should be evident if a plot is made of col- 
umns П and III in the extended factor matrix. 

The normal A defines the co-ordinate plane OAB of a simple structure. 


2 


FIGURE 1 


The direction cosines of A may be denoted X, Az, As, and, hence, the equation 
of the plane OAB may be written in normal form as 


(2) Arti + № + A323 = 0, 


where 2; is the projection of any point (a1, 22, хз) of the plane on the J-axis. 
Since the plane PP is tangent to the unit sphere and orthogonal to the 
I-axis, all points in that plane have unit projection on that axis, so that 
z,—1. The equation of the line of intersection AB of the two planes is, 
therefore, 


(3) M + Aste + A313 = 0, 


when referred to orthogonal axes in the plane PP which are parallel to 
II and III. 
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When a plot is made of columns II and III of Table 1, we have Figure 2 
where the points are clearly arranged in three intersecting lines. The enn. 
tions of these lines were determined graphically, and they are as follows: 
Trace of X-plane .58 — zs + 3215 = 0, 
Trace of Y-plane .55 + 8072 — 23 = 0, 
Trace of Z-plane — .71 + .83%2 +23 =0. 


These equations correspond to equation (3), and it will be seen that the con- 
stant term of this equation is the coefficient of x; in equation (2). 


Table 1 


Extended Factor Matriz Fs 


Test De I п III 
1 E 1.000 209 
2 Ly 1.000 —.905 
3 1. 1.000 .752 
4 1 1.000 —.510 
5 1 1.000 .609 
6 Ts 1.000 - 182 
T 1. 1.000 —.814 
8 T. 1.000 .377 
9 Т: 1.000 —.847 
10 jm 1.000 —.402 
11 15 1.000 .490 
12 Ly 1.000 —.106 
13 1. 1.000 .163 
14 1: 1.000 — .863 
15 1.5 1.000 . 823 
16 E 1.000 ‚176 
17 К 1.000 —.086 
18 in 1.000 .266 
19 1.4 1.000 —.926 
20 1. 1.000 735 


ables us to determine the direction num- 


The plot shown in Figure 2 en 1 { 
е. When these are normalized, we 


ers of a normal to each co-ordinate plan 
have the direction cosines of each reference axis Ap. 
The successive steps in the numerical work will now be described. On 


Figure 2 the three traces are denoted the X-, Y-, and Z-planes, as shown. 
The coefficients are recorded in the S-matrix of Table 2. In this case the 
Matrices S and L are identical. The columns of L are normalized as in the 
Previous work, and the multipliers 0,» are determined. Applying these mul- 
tipliers to the columns of L, we get the matrix A, the columns of which show 
the direction cosines of the three reference vectors Ap. It will be seen that 
this matrix A is the same as Ao: of Table 6 of chapter x, which was obtained 
after three rotations. The slight discrepancies are due to the fact that the 
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two solutions were obtained graphically. The columns X, Y, and Z of A in 
Table 2 correspond to the columns В, С, and A, respectively, in the ma- 
trix Ao, of Table 6 of chapter x. The order in which the reference vectors 
are listed in the columns of A, or of the factor matrix, is, of course, imma- 
terial. 

The oblique factor matrix V of Table 2 was obtained directly from the 
given factor matrix Fo of Table 1 in chapter x by the transformation A in 
Table 2. The matrix A'A = C shows the cosines of the angles between the 
reference vectors, and the matrix T shows the direction cosines of the pri- 
mary vectors Tp. Their angular separations are shown in the matrix ТТ”. 


27 


Ficure 2 


This is the matrix Rpa which gives the correlations between the primary 
factors Tp. These correlations are the same as the correlations shown in 
Table 8 of chapter x. The slight discrepancies are due to graphical methods. 
For example, the correlation 4-.105 in one table corresponds to the corre- 
lation +.112 in the other table. 

We have shown here an application of the method of extended vectors to 
a problem in three dimensions. Since this method consists in taking three- 
dimensional sections of an r-dimensional configuration and since r—3 in 
this case, the whole configuration as to its co-ordinate planes is defined here 
in one diagram, Figure 2. When r exceeds 3, it is necessary to make more 
plots, but not so many as in the method of plotting two-dimensional sec- 
tions described in the previous chapter. 
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The advantage of the method of extended vectors is that traces can be 
drawn anywhere on the plot, since they need not be radial, as in the two- 
dimensional method. A streak of points anywhere on the diagram can be 
brought into a co-ordinate plane. The number of rotations is usually much 
smaller than with the two-dimensional method, but the extended form of 
the factor matrix must first be computed. The method of extended vectors 
is not applicable when the configuration is of such a character that no cen- 
tral vector exists on which all the test vectors have appreciable positive 
projections. It is sometimes possible to use the method when all but a few 
of the test vectors have positive projections on some central axis, but special 


Table 2 
Factor Matriz V 
Matrix S= L X y 2 х Ү 2 
Tl э de — 1| .969  .003 —.003 
11 | —1.00 “30 .83 2 025  .939 005 
Tes Hl 32 —1.00 1.00 3 007 .023 957 
: : 4 2 .727  .000 
ур | 1.4388 1.3925 2.1930 5 87 .005 844 
УЕ | 1.1995 1.1800 1.4809 Ө) 012 à .399 — .T94 
D. "5337 8475 .6753 7 719 .512 —.025 
dep “ 8 S50 .016 384 
ا‎ 9| -.011 .760 444 
سک‎ 10 615 .623 —.008 
ранча 11 668  .008 38 
Matrix А X Y 2 12 | —.007 .610 628 
= 13| .956 .034 —.013 
1 483 466 479 14 | —.024 .921 063 
I | — .834 254 560 15 .028 —.019 937 
A, III 267 — .847 675 16 .282 .362 704 
p" : 17 .644 .400 273 
„== Е 18 947 —.032 008 
— — 19 .072 .906 —.040 
А 2 20 ‚004.032 .948 
Matrix C= AA x } 4 ieee te A 
N 1.000 
у | —..213 ‚999 -— 
съд z| — 1055 = -206 
Matrix 1” 
tmp 
Matrix TT’ x y ё 
5d 1.000 
; а .000 
о y s 1-324 0 
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adjustments are then necessary to write the final rotated factor matrix for 
the whole battery. 


The method of extended vectors for n dimensions 

The principle that has been described for a three-dimensional example 
will now be applied to a fictitious five-dimensional example. We start this 
problem with a simple structure, shown in Table 3. Here we have twenty- 
five variables, and arbitrary factor loadings have been recorded in this fac- 
tor matrix for the five columns shown. This matrix may be denoted F,. 


Table 3 
A Fictitious Simple Configuration Fx 

Pi Pe Pi P, P; he 

1 .0 .3 .0 .0 .8 -73 

2 0 9 0 0 0 .81 

3 .5 E .0 .5 .0 .75 

4 .5 ae .0 .0 0 ‚7% 

5 .0 :8 3 .0 ut .67 

6 .0 E! 6 4 .0 .68 

7 .0 .0 8 0 .0 .64 

8 .0 .5 T .0 .0 ‚74 

9 .0 .0 .6 .6 .0 .72 

10 .0 .0 4 6 5 .77 
11 .6 .0 .5 E! .0 A 
12 7 :8 .3 .0 .0 .67 
13 0 .0 .0 .0 9 .81 
14 0 .8 .0 E .0 .80 
15 .6 .0 5 .0 3 .70 
16 «D .0 .6 .0 .0 .72 
17 .0 .0 E .0 5 -74 
18 E! .0 .0 .0 4 .65 
19 .8 aly .0 .0 .0 .64 
20 .0 .0 .0 6 6 .72 
21 20 .0 .0 .8 .0 .64 
22 .3 .0 .0 .4 V .74 
23 .0 E .0 .5 .5 .75 
24 ot .3 .0 .0 .3 .67 
25 E .0 .0 8 .0 78 


The communalities are also recorded. In Table 4 we have an arbitrary or- 
thogonal matrix Azo, which transforms Fz into the factor matrix Fo of Table 5. 
We then have the relation 


(4) Fu = Fo. 


We start the problem with this matrix as given, and our problem is to ana- 
lyze it in such a way as to rediscover the simple structure of Table 3, which 
will be regarded as unknown during the computations. 

The first step is to extend the test vectors of Table 5. The last column of 
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that table shows the reciprocal of the first-factor loading by equation (1). 
Multiplying each row of F by the corresponding value D., we have 


(5) D.Fo = Ev; 


and we then have the extended factor matrix Eo of Table 6, in which each 


entry in the first column is unity. 
There are four columns that can be plotted to show the configuration, 


and these give six diagrams. Only three of these diagrams are shown in 


Table 4 


Ап А rbitrary Orthogonal Transformation Azo 


Table 5 
A Centroid Matriz Fo of the Р ictitious Test Battery 


1 п ш Iv Y De 
Z iss  .520 .325  —.224 1.9608 
480 536  —.070 — .378 2.7174 
474 —.026 —.304 —.084 1.5337 
"306 .370 | —.475 .153 1.9608 
1973 .373 .320 .023 1.6779 
140 —.209 166 452 1.6367 
Сыз -.300 102  .542 2.7983 
— 030 035 1048 .085 1.9342 
1074  —.557 .261 1993 1.8657 
_ 059  —.263 .454  —.182 1.4514 
1 2:083 —464 —.312 1047 1.4925 
1 L 070 ‘002 —.569 132 1.7544 
1 = "394 1394 394  —.394 2.2936 
1: “O54 “056 1913 1.9763 
1 —ün -.304 ‘038 1.5699 
15 ES 980  —.422 (238 1.8657 
16 а —.044 .309 .256 1.8018 
17 Е) 270 —.026  —.419 1.9305 
18 a ‘074 —.666  —.226 2.7933 
19 e — 070 MAT —.447 1.7889 
20 — 1443 "946 —.246 2.7933 
21 057 1180 —.514 1.5337 
22 zi 240 "330 —.162 1.4995 
23 246  —.476 —.203 1.7212 
2. E 471  —.004  —.330 2.0325 

9 
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Figures 3, 4, 5, and 6, since these are the only ones that were used for de- 
termining the initial locations of the reference vectors. 

The successive computations are shown in the formulae for each part of 
Table 8 and subsequent tables. It will be seen that some of the matrices 
have an added column for the first centroid factor, which was used in writ ing 
the extended factor matrix. The new notation is for the matrix M, which is 
identical with A except for the added column for the central axis (I). At 
any stage in the rotations, the matrix A can be written by merely omitting 


Table 6 
Extended Factorial Matriz Ey 


I II III LV у 
1 1.000 — .369 1.037 ‚637 — .438 
2 1.000 1.329 1.457 — .207 1.027 
3 1.000 .727 — .040 — .466 — .129 
4 1.000 . 600 .725 = 981 . 300 
5 1.000 — .458 .626 ‚537 ‚039 
6 1.000 .295 — .842 .272 . 740 
7 1.000 — .964 — .838 .285 1.514 
8 1.000 — .058 .068 .093 1.325 
9 1.000 .138 —1.039 .487 .416 
10 1.000 — .086 — .382 ‚659 = .192 
11 1.000 — .124 — .693 — .466 .070 
12 1.000 = he: .004 = 3098 .232 
13 1.000 — .904 . 904 . 904 — .904 
14 1.000 1.296 .502 -111 .421 
15 1.000 == 208 = ki = A4 .060 
16 1.000 — .647 — .522 mr .444 
17 1.000 — .939 — .079 . 557 .461 
18 1.000 es XOT .521 — .050 = <809 
19 1.000 — .330 = 207 —1.860 — 6081 
20 1.000 .125 = «lee . 800 — .800 
21 1.000 1.237 —1.237 . 687 — .687 
22 1.000 = «198 ‚087 .276 — .788 
23 1.000 .493 .358 .493 — .242 
24 1.000 = То .423 — .819 — .349 
25 1.000 .811 .957 .008 .671 


the first column from the matrix M. In the method of extended veetors the 
matrices S, L, H, and C have the additional column for (Т). 


The new B-plane 

Figure 8 shows the plot for columns II and IV of Table 6. A unit vector 
is indicated by an arrowhead on the II-axis, and another unit vector is 
shown similarly on the ZV-axis. These are the given unit reference vectors 
(IT) and (IV). A line has been drawn through a streak of points to repre- 
sent the trace of one of the planes. It is not necessary that the traces should 


1 The notation “(7)” will be used to denote а unit vector in the axis Г, and similarly for 
the other orthogonal reference axes. 
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bound the configuration as in the present case. A normal is drawn through 
the origin, as shown by the dotted line. The direction of the normal is 
shown by the arrowhead. The direction of the normal is so chosen that the 
projections of most of the points, which represent test vector termini, will 
be positive. Since this normal is closer to the Z-axis than to the other axis, 
we denote the trace the B-plane, and the normal is denoted B4. The sub- 


ha 


1 


script of В! represents a new location, and the prime indicates that the new 
vector is a long vector that has not yet been normalized. 

Since the new vector is closer to (JZ) than to (JV), we shall describe it as 
equal to the unit vector (IZ) plus corrections in the directions of (ТҮ) and 
(I). The vector (IJ) is then a base vector for Bj. In Figure З we draw the 
normal Bj so that its terminus is directly above the terminus of the unit 


FIGURE 3 
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vector (II). The new long reference vector B2 can be deseribed by the 
vector equation 


(6) By = (II) + айу) +41), 


which requires only two numerical values to be found from the graph, 
namely, d and i. When found, these values are recorded in the B; column 
of the matrix So, in Table 8. 


FIGURE 4 


The value of the coefficient d is read directly from the graph. It is mere- 
ly the vertical distance from the terminus of (IT) to the terminus of Bj on 
Figure 3, namely, 0.23. These diagrams are ordinarily drawn on cross-sec- 
tion paper, where finer gradations are available than on the figures of this 
chapter. 


The absolute value of the coefficient 4 is read directly as the intercept of 


e 
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the trace on the same axis as the base vector (IT). This intercept is 0.79. 
The coefficient i is taken as positive if the origin and the new vector are on 
the same side of the trace, as in this case. The coefficient 7 is taken negative 
if the origin and the new vector are on opposite sides of the trace. 


[О] 


x 


FIGURE 5 


i 1 ve nd i on the graph, we can writ 
Having read the two numer ical values d and i graph, a d 
] form, namely, 


the vector equation (6) in numerica 


(0 Bi = 10001) + 23V) + 790), 
and these coefficients are written in the column B! of the matrix So, in 
Table 8. 


The new C-plane 
In Figure 4 we have the plot for | 
Matrix Hy of Table 6. The two unit 


r columns II and Ш of the extended factor 
vectors (JI) and (117) are shown by ar- 
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rowheads in the JI- and 7//-axes, respectively. A trace has been drawn 
through or near eight points. The normal is nearer 7/7 than 77, and hence 


the trace is labeled the C-plane. 
Since the unit vector (JIT) is the base vector for this diagram, we draw 


Ficure 6 


the normal C so that its terminus has unit projection on (JII), as in the 
previous figure. The new vector C; is described by the vector equation 


(8) C; = (ПІ) + 6I) +70). 
The coefficient b is read directly from Figure 4 as the distance between the 


termini of (III) and C4, namely, —.52. The intercept of the trace on the 
III-axis has the absolute value .35. It is taken positive, since the origin 


THE METHOD OF EXTENDED VECTORS 287 


and the vector С? are on the same side of the trace. Hence we have the vec- 


tor in numerical form, 
(9) Ch = L00UII) — 5201) + 350), 
and the coefficients are recorded in the column С; of So: in Table 8. 


The new E-plane 

In Figure 5 we have the plot for columns II and V of Table 6. Here a 
trace has been drawn through, or near, twelve points. The normal is drawn 
through the origin to E? so that it has unit projection on (V), which is taken 
as a unit base vector. The vector equation of E; then is 


(10) E} = (V) + 00D + D, 


where b and i are to be determined from the graph. The value of b is read 
as —.70. The absolute value of the intercept of the trace on the V-axis 
is .35. It is taken as positive because the vector Æ, and the origin are on 
the same side of the trace. Hence we have the vector equation in numerical 


form, 
(11) E; = 100(7) — 7001 + 35, 


4 


the coefficients of which are recorded in column E, of So, in Table 8. 
The new A- and D-planes 


Figure 6 shows an example of 
termined. In locating two planes 


a graph in which two new planes are de- 
on the same diagram, one avoids drawing 


traces that are parallel, for reasons to be discussed. One trace is drawn 
through seven points. Such а configuration is useful in locating the simple 
Structure not only because a number of points lie in а straight line but also 
because they extend over some length on the diagram. In the case of two 
Alternative traces with the same number of points, one prefers the one that 


has AT 
as more length on the diagram. ; p ў 
Another HTN is drawn through about thirteen points, and a normal is 
drawn through the origin. Both normals are drawn in the direction of the 

8 i ill have positive projections. 


COnfigurat; ^ at most of the test vectors W ; : 
guration, so that mos ane parallel to (IV), this trace is denoted 


Since t] is tre 
шщ mal to this trace 1 м 
the 7 лы pre trace might have been denoted the E-plane, but 
Such а plans has already been chosen, so this one 1s denoted the A-plane, 
hus locating f VM lanes, 4 to E, inclusive. "n 
The e EE db Dia is drawn through the origin and extended so 
that it has unit projection on the base vector (IV). The vector equation for 
the normal then is 


(12) pi = -UV) + 60) + i), 
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where e and i are to be read graphically. The value of e is —.26, and the 
absolute value of û is .54. This is taken positive because the normal D; and 
the origin are on the same side of the trace. We then have the vector equa- 
tion in numerical form, 


(13) D} = —1.00(IV) — .26(V) + 54, 


the coefficients of which are recorded in columns D; of So: in Table 8. 

The normal to the A-trace is drawn through the origin to the point А; 
with unit projection on (IV), which is arbitrarily chosen as a base vector. 
The vector equation for Aj can be written 


(14) As = (IV) + e(V) + ill). 


The numerical value of e is —.97, and the absolute value of i is 1.20, 
which is taken positive because 4; and the origin are on the same side of 
the trace. We then have the vector equation in numerical form, 


(15) A} = 1.00(1V) — .97(V) + 1.20(1) , 


the coefficients of which are recorded in the А; column in So, of Table 8. 

It so happened that all the increments 7 for the vector (7) were positive 
in this set of four diagrams. In each of the subsequent rotations there will 
be cases of negative increments for (I), as may be seen from the diagrams 
and from the first row of the matrix S in Tables 9, 10, and 11. 


Independence of planes 


When the traces have been drawn for the planes, it is desirable to ascer- 
tain, before proceeding with the computations, whether the planes are 
linearly independent. It sometimes happens that the same plane is repre- 
sented by streaks of points on several diagrams. If these traces were identi- 
fied as different planes, one would discover in the subsequent computation 
that the two corresponding normals had an angular separation with a co- 
sine of, say, .80 or .90. The result might be that one of the common-factor 
dimensions had been lost. In order to avoid the annoyance of the computa- 
tional adjustments that would then be necessary, it is well to inspect the 
planes for linear independence. This is not necessary in the later rotations 
where the planes are easily identified from one rotation to the next, but it is 
advisable to make an inspectional check on linear independence at the start 
of a factorial analysis. The check will be described here only for the initial 
location of the reference vectors. 

In Table 7 we have five columns, one for each of the five normals that 
have been chosen. In column A there is recorded a plus sign for each test 
variable that has a large positive projection on the normal A}, a zero for 
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each test that has nearly vanishing projection on that normal, and a blank 
space for those tests whose projections are neither conspicuously high nor 
vanishingly small. In some of the columns there are recorded minus signs 
for those tests which have conspicuously strong negative projections. 

For every pair of columns, there must be differentiation as to the plus 
signs, the minus signs, and the zeros. In the present case we have chosen 
the five normals so that they are linearly independent. If we had chosen 
two normals with small angular separation, they would have had the same 


Table 7 


Projections on New Planes 
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ive projections, the same tests with strong negative 


tests with high posit kit NA 
Saturation ui the same tests with small projections. That would indicate 
the fact derer dependence, and one of the normals could then be discard- 


ed in favor of a new one which should be linearly independent of the other 
four normals. If such a situation arises, it is a useful procedure to select the 
tests which have no plus signs or minus signs and to find a trace on whose 
normal the missing tests would have significant projection, either positive 
or negative. That would, in general, insure the recovery of all the dimen- 
sions of the common-factor space. One way to avoid the loss of a dimension 
is to designate a new reference vector by the fixed orthogonal axis to which 
it is the closest, but this procedure is not always feasible. 
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Alternative method of writing the S-matrix 

A variant of the method that has been described for Figures 3, 4, 5, and 6 
may be preferred by some students. One writes the equation of the trace in 
the form 


(16) аз хз + ag: +d =0, 


which might represent the A-plane determined from a diagram of two col- 
umns Il and III. If most of the points of the configuration are on the 
same side of the trace as the origin, then the constant term in (16) should be 
positive. If the constant term in (16) is negative, the signs of all the terms 
are reversed. If most of the points in the configuration are on the opposite 
side of the trace from the origin, then the constant term must be negative. 
The three coefficients are then recorded in the appropriate column of the 
S-matrix. Other variations of these procedures can be easily devised to 
suit individual preferences. 


Computations for Table 8 

When the matrix So: has been written by inspection of the graphs, the 
remaining computations of Table 8 follow the formulae given and in the 
order given in the table. In the initial location of the reference vectors the 
matrices So: and Loi are identical because the matrix S describes the new 
vectors in terms of the fixed orthogonal frame. From the matrix 
Lo, we obtain the diagonal entries of D; as shown. Then we determine Hoi, 
which is in this сазе identical with Mo. For subsequent rotations they will 
be different. The matrix Cı shows the cosines of the angular separations be- 
tween the reference vectors. These cosines are recorded on the diagrams 
ation, as shown in all subsequent figures. The new oblique 
factorial matrix Ё is computed either from Hoı or from My. In practice 
it is not necessary to write the column I in the extended factor matrices. 
The column is here given explicitly to show the complete result of the in- 
dicated matrix multiplications. The factor matrix E; is used for plotting 


the next set of diagrams. 


reference 


for the next rot 


The first rotation т 
airs of columns of the factor matrix E, are plotted on as many 


е are inspected to select the first rotation of the reference 
f the ten diagrams were used, and they are represented 
in Figures 7, 8, and 9. Figure 7 shows a type of plot that occurs rather fre- 
quently in factor analysis, which ш ап гта bi анон of two 
axes, Ау and Cy through a large angle. n all t tese IAMS we record in 
the torner of the diagram the cosine of the angle separating the two ref- 
erence axes that are represented by the po» ^ ^ s os deviate 
markedly from zero, they are used in making a choice of rotation. From 


The ten p 
diagrams, and thes 
axes. Only three 0 
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this diagram we obtain the entries in columns Ag and С; of "ae Sr td 
in Table 9. We have here an example of a negative correction ort io ve 
tor (I) in the column C4. The negative sign of this correction is determinec 
by the fact that the vector С; and the origin are on opposite sides of ү 
trace for the new C-plane. The procedure is the same as that bescribed for 
the previous diagrams and for Table 8. 


e 


COS =/94 


Fiaure 7 


In Figure 8 we have the plot for B and D, on which h 
trace for the new D-plane, with a slight adjustment. In 
the plot for columns С and Е, which indicates 
ment in the E-plane, The cosine of the angul 
Ci is +.325. The positive sign of this cosine 
ence vectors should separate if they are to be 
nal. In the present case the vector E 


as been drawn the 
Figure 9 we have 
a rather compelling adjust- 
ar separation between X’ and 


1 
indicates that the two refer- 


orthogonal or nearly orthogo- 
1 moves away from the vector C, to 
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the new position £3. The positive sign of the cosine and the appearance of 
the diagram show that the two vectors E and C will be orthogonal, or 
nearly so. In this rotation no adjustment is made in the B-plane. This is 
represented by the entry of unity in the B column of S;» in Table 9. Hence 
B= By. 
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The second rotation 

Pairs of columns of the oblique extended factor matrix E» are plotted on 
ten diagrams, three of which were used to select the adjustments for the sec- 
ond rotation of the reference axes. These three plots are shown in Fig- 
ures 10, 11, and 12. 

Figure 10 gives two clear traces of the planes. On the vertical axis we 
see a large collection of points near the origin and no points elsewhere on 


FIGURE 8 


— == = = ae 
Шем = - A 
әд" AI 
I98 — = ПІ 
SER, — =: ЖУ П 
Str оо” 000i I 
“ч ‘a p 1 
000°T GSO’ imit — eim Я = 
0001 9&0’ 10° a = ар 
000°т FET Ü 
000 I Я | 
F SFG 17 
000°T 1 sen @ 
= тї” = 1) 
a a 3 a [ 1 FR 0001 ig 
or tp 
ele er - fle 000 T 1 
onn = 
ч ‘a 9 " 1 
+07 — соё 029'1 боот == 
ИПИ БИЯ GIL’ — 00071 ‘as = ұн 
ogo’ Sos" SAI б 000°T 
epist 8 oe, 3 
Bem m. m Pm 
000: — Ол GIS 00-1 O0IOF'I — O000'0 олет 
61 $00: со” 0001 ELE 8666" TEGE 
FOL’ Mou 690)" 0007 D 
F00: КАН 9107 = 000°T - | - n 
GIT коо — ОО — 000^ I Б! J AT 
5807 ei £00 — 000 T А 290:1— iu 
WI FOE 089 2087 000°T = Bie осо. Il 
00 = 1605 820° 000°T 019 609 000°T I 
820° — 000:1 i 
800° — 000°T a 2 іЯ SW F 
во. — 000. гт 001 
10 — 060 — SST" 000°T pr E 
2 201 160° — 0001 SUAE ир 
£10: IF: — = 000°T 
0с0` 500° OSs" 000°T 00^ 1 1 
60° СЕ” Z 00071 00°T @ 
OTI — 000°T ек — 001 96° 1— 1) 
£I 0007 T er = 00-1 i 
625" [M 66€ I = 00071 907T 00:1 к 
90" ecoT GEF = 00071 Št 86° — EE 00°T 1 
а 2 ч Ld І ia Юю ig > | 
Я атыруу 


"HQ = UI 


G'g ‘4 soanDig wuosf. suoivnduto,) 
6 2190 ], 


eot 


THE METHOD OF EXTENDED VECTORS 24 


the vertical axis. Two points, namely, 2 and 4, are in a radial line, and these 
are combined with the points near the origin for the trace of the new 4- 
plane. The corresponding entries are made in the matrix S»; of Table 10. 
The other trace on this diagram is also clearly indicated. There are no 
points on the horizontal axis of this diagram, except those which are at the 
origin, and consequently there is no configuration to hold the reference 


e 


COS = 325 


| | | L 


Ficure 9 


normal B in its given position. On the other hand, there are seven points 
in the first quadrant which are radial, and these are used in selecting a newly 
adjusted position for the B-plane. In each case the plane is identified by 
adjustments that are less than 45°. The corresponding entries are made in 
the matrix Ss of Table 10. 

In Figure 11 we have a small adjustment in the D-plane as shown, and 
in Figure 12 there is indicated a small adjustment in the E-plane. These 
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diagrams indicate very clearly that the configuration is closing in toward a 
simple structure. No compelling adjustment was found for the C-plane in 
this set of diagrams, and consequently there is only one entry of unity in 
the C-column of matrix Səs, according to which the new reference vector 
Chis collinear with the given reference vector Сї. 


8 


COS = «580 | 


ha eed | 


FraunE 10 


les in this set of diagrams shows one large cosine, 
namely, A B, which is 4-.580. If the simple structure is orthogonal, or near- 
ly so, we should expect the two reference axes, A and B, to separate. In- 
spection of Figure 10 shows that, when we follow the configuration, the two 
new reference axes, As and Bi, do separate, as shown by their new locations 
in the dotted lines. Their given positions are plotted as orthogonal in the 
figure, but their new separation is an obtuse angle. Further rotations should 


Inspection of the ang 
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give only small angular adjustments. The computations for this rotation are 
indicated in Table 10, and the result is the extended factor matrix Ез, which 
is used for plotting the next set of diagrams, 


The third rotation 


The third rotation is determined by the traces that 


are drawn on Figures 
13-17, inclusive. The procedure is the same as for the 


rotations selected on 


HH 


Ficure 11 
the previous diagrams, е 
E-plane. Figure 17 indic 
and Figure 14 


xcept for the combin 
ates а small 
indicates а sma 
rections are limited to small an 


gular adjustments of ap ith traces through 
the origin, then two or more adj 


he same rotation. 
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Figures 14 and 17 show the combination of two adjustments of the E-plane 
in the same rotation. In Figure 17 the long vectors that are brought into 
the plane are 20, 21, and 25, and these are near the origin of Figure 14. 
Hence a small adjustment in the E-plane on Figure 14 does not affect the 
factor loadings on E of these three tests. On the other hand, a small adjust- 


т 


©о$ = - /6/ 


FIGURE 12 


ment of the E-plane in Figure 14 involves test 2, which is near the origin 
in Figure 17. The two adj ustments are therefore said to be independent. If 
tests 20, 21, and 25 had high projections on both A and B, then the two 
adjustments from these diagrams could not be combined because they 
would result in an overcorrection. In making the adjustments from Fig- 
ures 14 and 17, it would not even be necessary to identify the numbers of 
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the test vectors in both figures because of a principle that is often se 
When one adjustment moves a configuration slightly out of a plane while the 
other adjustment moves it slightly into the plane, then the two adjustments can 
be combined with assurance that they are independent without identifying the 
test vectors in both diagrams. In the present example the adjustment shown 


2 


Figure 13 


in Figure 17 tends to move the plane down from the configuration, whereas 
the adjustment shown in Figure 14 tends to move the plane up into the con- 
figuration. Hence the two 


adjustments can be combined with assurance 
that they are independent. The adjustment in Figure 14 will produce a 


small negative factor loading for test 14, whereas the adjustment in Fig- 
ure 17 will produce a small positive factor loading for test 14. The result 


Re 
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should be a negligible factor loading for this test when the two adjustments 


are combined. 

The adjustments selected in Figures 13-17, inclusive, are recorded in the 
matrix S3, of Table 11. The remaining computations for this table proceed 
according to the formulae shown and as described for previous rotations. 


ze 4. 


COS = -030 


FIGURE 14 


The result is shown in the oblique extended factor matrix £4 of Table 11. 
The matrix C4 gives the cosines of the angles bet ween the reference vectors, 
as well as the projections of these vectors on the unit centroid vector (I). 
It is seen from this table that the reference vectors are nearly orthogonal. 
In dealing with an oblique factor problem it should be recalled that the 
matrix C gives the cosines of the angles between the reference vectors. The 
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correlations between the corresponding primary factors are then deter- 
mined by the formulae that have been described in chapter x. 

In Figures 18 and 19 we have the complete set of ten diagrams which 
show the configuration in relation to a set of reference vectors, and there is 
no further indication of adjustments in the reference vectors. This set of 
diagrams is therefore accepted as final for this problem. The matrix M o, of 


Сөз 


FIGURE 15 


jsformation which carries the given extended factor 
nded factor matrix 24. The same transformation 
trix Fo, that was obtained by factoring, into a 
present a set of orthogonal axes in 


Table 11 gives the trat 
matrix E, to the final exte 
carries the given factor ma 
factor matrix Va, which happens to re 
the present problem. We then have 


(17) Е Ме = E4, 
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which is an alternative for 


(18) Palan = JR. 


We also have 


(19) Foda = V4. 
Table 12 
Vi = Podu 
1 B с D E 
1 —.016 -319 -008 
2 —.012 .899 -006 
3 493 .502 —.015 
4 —.010 .715 —.012 
5 —.010 .315 -307 
6 404 -385 -602 
7 012  —.005 -800 
8 004 .495 -704 
9 -609 —.019 598 
10 .599 —.006 401 
ll .406 005 E 
12 —.001 320 277 
13 —.013 022 .006 
14 .390 788 -005 
15 -001 023 -480 
16 .007 015 579 
17 .003 007 . 704 
18 —.011 030 —.009 
19 —.001 026 —.029 
20 .592 001 .002 
21 .801 — —:020 —.002 
22 .390 016 —.007 
23 486 499 .005 
24 —.010 330 —.021 
25 .800 —.011 —.013 


This is the formula shown at the top of 
trix V4 for the twenty-five test vi 
matrix is used for purposes of interpretation. 

In the present fictitious example we 
rediscovered the factorial composition о 
That factorial composition is shown in Table 3 
two tables are nearly identie 


Table 12, which giv 
are interested to find 


al. In making the comparison, we 


es the factor ma- 
ectors with normal length. This factor 


that we have 


f the matrix with which we started. 
; and it will be seen that the 


must identify 
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the five columns. The columns Р; to P; of Table 3 correspond to columns 
D, B, E, A, and C in Table 12. The reason for the arbitrary rearrangement 
of the columns is that in locating the traces of the planes in the diagrams 
and in giving letter identifications to the planes during the rotations we have 
no way of knowing which planes are emerging during the successive rota- 
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tions. Therefore, in making several factorial analyses of comparable test 
batteries, the order in which the primary factors appear in the successive 
columns of the factor matrix is entirely arbitrary. 

When the test vectors are identified by their test names, as well as by 
their code numbers, one learns to identify early in the rotations the well- 
established primary factors. In some problems that are concerned mainly 
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with methodology, the planes are given code numbers, which are relied 
upon exclusively for the location of the traces on the diagrams. In. ex- 
ploratory factorial studies it is often found that the primary factors 
completely determined in some of the diagrams because the test b 
in exploratory work seldom reveal 


are not 
atteries 
a complete simple structure. It is then 


E T4 
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FIGURE 17 


advisable to use all the informat 


as well as the psychological nature of the tests, to determine the со 
planes, which are then interpreted as in the nature of 

ing the underlying parameters or primary factors, These must then be in- 
vestigated with further factorial studies to determine whether one or more 
hypotheses concerning the postulated functional unities can be sustained, 
The purpose of this chapter is to show that when a clear and complete 


ion available, including the configuration 


-ordinate 
hypotheses concern- 
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simple structure exists, as in the present fictitious example, the rotational 
methods that have been described enable us to identify the underlying 


al orthogonal test bat- 
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CHAPTER XII 
THE SPECIAL CASE OF UNIT RANK 


The one-column factor matrix 

If a correlation matrix can be accounted for by a single common factor, 
then we have a factor matrix of one column, as shown in Table 1. The fac- 
tor loadings can then be denoted with a single subscript, a; for test j. By 
the fundamental factor theorem we have FF’ = f, so that the diagonal ele- 
ments become aj =h}. Each communality Aj is then simply the square of 
the single-factor loading, namely, aj. In dealing with actual data it is not 
often that one can account for the correlations by a single common factor, 


Table 1 

5 * Factor 

Correlation Matrix Matrix 

D ee 4 % 2 

aî ns m Тин : а 
2 na uio fu Чи 2 а 
3 | ro aj ти ы 
4 nu та 08 E A 
5 ns Tts Ts T6 a t 


but this ease is of historical interest because it was the case to which stu- 
dents of factorial theory devoted almost their entire attention for a quarter 
Of a century after Spearman's first provocative paper on this subject in 
1904.* We shall consider the single-common-factor problem as a special 
case of the multiple-factor problem instead of by the statistical methods 
formerly used. We shall refer to some of the controversial issues about the 
Single-common-factor problem that were debated formerly, and they will 
be discussed here in the light of multiple-factor theory. 
l'or the ease of a single common factor we have 


(1) Tjk = Qr, 


Where a; and a; are the single-common-factor loadings of tests j and k. In- 
Spection of this equation shows that pairs of columns, Ё and 1, must be pro- 
Portional. We then have the theorem that if a correlational matrix is of unit 

* “Correspondence between General Discrimination and General Intelligence,” Part 
III of “General Intelligence Objectively Determined and Measured,” American Journal 


of Psychology, XV (1904), 268-72. 
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rank, then all pairs of columns, or rows, are proportional. The converse of 
this theorem is also true, for if all pairs of columns are proportional, then 
all minors of second order or higher vanish, and hence the rank must be less 
than 2. The trivial case of rank 0 is here of no significance. The case in 
which the rank is 0 is, of course, identified by the fact that all the inter- 
correlations are 0. That is a case of no scientific interest. We have, there- 
fore, the theorem that if all pairs of columns, or rows, of a correlational matrix 
are proportional, then the rank of the matrix is 1 or 0. 

One of the earlier criteria used by Spearman for showing that the inter- 
correlations of a set of tests could be accounted for by a single common fac- 
tor was the proportionality of the columns of coefficients. Because of sam- 
pling errors the proportionality was not exact When experimental data were 
used. A later criterion of the degree of correspondence between pairs of 
columns was the correlation between them. In general, if two columns are 
proportional, their correlation is +1 or —1. The degree of proportion- 
ality was expressed in terms of the coefficient of correlation. We then have 
the theorem that if the correlational matrix is of rank 1, then the correlation 
between pairs of columns, or rows, is +1 or —1. The converse of this theorem 
is not necessarily true, If the correlation between pairs of columns is 4-1 
or —1, the columns are not necessarily proportional. A specifie case which 
disproves the converse is that in which a constant is added to each element, 
of a column. A linear plot is then obtained which does not pass through the 
origin. The correlation is +1, but the columns are not proportional. 

Spearman’s former use of the intercolumnar criterion depended on the 
converse, in that the high correlation between columns was the basis for 
the inference that a single factor was sufficient to describe the intercorrela- 
tions. While the intercolumnar correlation criterion is demonstrably fal- 
lible, it would be a rare situation in which a set of mental tests would satisfy 
the criterion of unit correlation, or very high correlation between columns 


when the rank is higher than 1. Hence Spearman’s use of the correlation 
criterion could be defended, 


| But another type of difficulty appeared with the intereolumnar correla- 
tion criterion. If all the coefficients in R are of the same order of magnitude 
mpling errors, then the dispersion of a col- 
1e sampling errors, and the correlation be- 
x \ ause of the restricted range of the entries in 
the correlation matrix. The proportionality would still be maintained with- 
in sampling errors, but the points in each correlation table would be so re- 
stricted in range that the correlation coefficient, would not reveal the pro- 
portionality. The intercolumnar Proportionality criterion is therefore su- 
perior to the intercolumnar correlation criterion. 

The limiting case of this effect is o 
in a correlation matrix 


and if these are overlaid with sar 
umn may be comparable with {} 
tween columns may be low bee 


f some interest, 


Tf all the coefficients 
are equal, then the proportionality criterion is satis- 
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fied, but the correlation coefficient is indeterminate. The proportionality 
criterion would give the correct inference, namely, that the correlation ma- 


trix is of rank 1. We then have 


Tjk = ајаг 


(2) Te = ака!, 
ти = ауа. 

But 

(3) rye = тл = ТЫ) 


and hence 


(4) ajay = ауа = аки, 
and 
(5) dj = а=. 


It follows from (1) and (4) that 
(6) q= а == Утук. 
This limiting case is represented in the theorem that if all the coefficients 
e وي‎ x . . 5 
rix in a correlation matriz are equal, then the matrix is of unit rank, and each 
test has a single-factor loading of V Tik- v. | 
If sampling errors are superimposed on this limiting case, the correlation 
between columns shows only the correlation between random errors. This 


correlation should be zero or near zero. The intercolumnar proportionality 
criterion is still valid, and it would be only slightly affected by the sampling 


errors in a finite test battery. T 
Spearman's use of the correlational, rather than the proportionality, form 
of the intereolumnar criterion was determined, probably, by the fact that 
the standard error of a correlation coefficient could be determined, whereas 
the proportionality form of the criterion would require the development of 
àn appropriate standard error formula. There does not seem to be any fun- 


damental difficulty in developing such а formula. А 
Since the diagonal values of the correlation matrix are frequently dis- 


cussed in factor analysis, it is useful to know the theorem that if a correlation 
matrix is of unit rank and if the diagonal elements are unity, then all elements 


tn the matrix are +1 or —1. нй Я 
It is sometimes useful to note a restriction on the numerical values in a 


single-common-faetor correlation matrix. It can be stated in the theorem 
that when the rows and corresponding columns have been reversed in sign so 
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that all the coefficients in a single-common-factor correlation matrix are positive, 
then the three side correlations of any third-order principal minor are such that 
any one of them is greater than the product of the other two. This property fol- 
lows directly from the proportionality of the columns and the further re- 


striction that the diagonals are positive numbers that do not exceed unity. 
We then have 


Tio _ T22 
тд Taz 
But тоо = А2 <1, so that 
712793 
Tis ы 


апа һепсе 
712793 < тз. 


If a set of three tests have intercorrelations that do not satisfy this restric- 
tion, then a diagonal value is greater than unity. Such a set of three tests 
can be a part of a correlation matrix of rank that is higher than 1. 


The number of tests for unit rank 


It is useful to inquire how many factors can be determined from the in- 
tercorrelations of any given number of tests or measures, Here we are con- 
cerned about the number of tests which are necessary to determine a single 
common factor, In a correlation matrix of n tests we have 


in(n — 1) 


independently determined correlation coefficients, because these include 
only the side correlations and not the diagonal or self-correlations, which are 


unknown. In Table 1 we seo that for rank 1 the diagonal elements are simply 
the squares of the factor 


or loadings unless we are given at 
e then have 


(7) = эй. 
ог 
(8) n>3 
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also three intercorrelations, namely, гу, 73, Tete The solution of this case 
can be found from the three intercorrelations, 


(9) тк = айк, 
(10) rj = ај, 
апа 

(11) ты = ами. 


Divide (10) by (11). Then 


(12) Tt ыб 
Ты @к 
So that 


Substituting (13) in (9), we have 
(14) E 


from which we have Spearman’s formula* for the correlation of test j with 


the single common factor, namely, 


dis PA E : 


of test j. The correlation rj between test j 

and the single common factor is the factor loading ау, so that тл = ал. 
The intercorrelations of three tests can usually be aceounted for by one 

common factor, but such is not always the case. Consider the two correla- 


In which a; is the factor loading 


Table 2 
25 ex, йй 1:85 
T J ED wo = 50 
'a5 .50 ps .35 50 -- 
5 (0) 


(a) 
(b) in Table 2, which have been used by Wilson and 
пе correlations between three tests cannot always 


tor. The correlation matrix (a) is of 


tion matrices (a) and 
Worcester to show that tl 
be aecounted for by one common fac 


* C. Spearman, The Abilities of Man (N 


Р. xvi, eq. (19). pu 
T “т i ts into Two G 

ТЕ. B. Wilson and Jane Worcester, «rhe Resolution ог = о General 

Factors,” pure of the Nationa ту of Sciences, XXV (January, 1939), 24, n. 3. 


ew York: Maemillan Co., 1927), Appendix, 


1 Acade: 
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rank 1 if we insert the communalities .07, 28, .89, which determine the 
single-common-factor loadings. The matrix (b) is of rank 1 if we insert the 
diagonals .098, .20, 1.25; but, since one of these is greater than unity, the 
solution is incorrect. Matrix (b) can be considered as a correlation matrix 
of rank 2, and there will be two general common factors. In this case the 
resolution into factors is, of course, not unique. 

The value of a; as determined by equation (15) is subject to fluctuation 
with the sampling errors of the three correlation coefficients in terms of 
which it is expressed. It is desirable to minimize this eff. 
average value for a; based on different pairs of tests k 
jis combined. With n tests, the number of ways in w 
ten for test j is the number of 
maining (n— 1) tests, 


ect by taking an 
andl, with which test 
hich (15) can be writ- 
pairs of tests that may be taken from the re- 
excluding test j. Hence the number of ways in which 


Figure 1 
(15) may be written is $(n—1)(n— 
for ascertaining all the 
If we have twenty tests, the: 
ferent ways in determining { 
the single common factor, 


2). The total number of formulae (15) 
single-factor loadings for n tests is 3n(n—1)(n—2). 

n formula (15) can be evaluated in 3,420 dif- 
he correlation of cach of the twenty tests with 


are required in order to determine a single 
common factor. It is of some i е nature of the indeterminacy 
when only two tests are available. Consider Figure 1, in which are shown 
the two test vectors, j and k. The correlation between tests j and k сап be ex- 
pressed as the scalar product of the two test vectors, во that 
(16) 


Tit = hihe cos фу; 


but the figure cannot be drawn un 
ly, the two vector lengths and th 
tion is represented in F 


iquely unless we know 
er angular sep 
igure 1 by the pair of 


three things, name- 
aration. The same correla- 
vectors j and k’, where the 


THE SPECIAL CASE OF UNIT RANK 265 


angle and the communality of &' are both altered so as to retain the same 
given scalar product or correlation. The case of one test determines an axis 
that may be defined as collinear with the test itself, but the problem of com- 
mon factors does not appear except for two or more tests. If equation (16) 
is interpreted for one dimension or common factor, then the cosine of the 
ге have two communalities which cannot be 


angle is, of course, unity, and w 
determined by a single correlation coefficient. We see, therefore, that two 


tests do not define a single common factor. 
Students of factor analysis who confined their attention primarily to the 


single common factor have used sets of four tests to determine the common 
factor, The reason for this is that, in general, three tests determine a single 
common factor uniquely, since there are three intercorrelations to deter- 
mine three factor loadings. Hence nothing is proved scientifically by doing 
it. A scientific demonstration of any kind must always be overdetermined 


Table 8 


to 


e оо 


ing. Hence the smallest number of tests 
be demonstrated is four. Of course, 
Jation matrix, the more convincing 
assuming that the 


by the data in order to be convine 
by which a single common factor can 
the larger the number of tests in the corre 
Will be the demonstration of а single common factor, 


variables are otherwise proper] i E T 
The case of rank 1 requires that all the correlation coefficients be positive 


or that they can be made positive by reversal of one or more of the meas- 
ures. Consider, for example, the correlation matrices in Table 3, which 
are both of ank 1. The absolute values are the same. The first matrix can 


be transformed into the second by reversing the signs of the second and 
then the fourth variables. Each reversal represents а reversal of a column 
and then the corresponding row: The reason for this arrangement for rank 1 


is that the test vectors are colline: y may be reversed in direction. 


y chosen. 


ar and the 


The tetrad difference “ 
Spearman’s later method was to evaluate what are called “tetrad differ- 


ences.” The tetrad difference is of the form 


(17) runin ТнТ = Ps 
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where k and l refer to two rows, while m and n refer to two columns of R. 
The four subscripts refer to as many tests, and it is implied that four sepa- 
rate tests are involved in the tetrad difference equation. Hence the tetrad 
difference is not written so as to include any diagonal terms of R. This is 
consistent with the fact that the communalities are unknown 
has shown that if only one factor is involved, then all the tetr 
in R vanish. 

The tetrad differences have a very 


. Spearman 
ad differences 


simple matrix interpretation. They 
are simply the expansions of second-order minors in the correlation mat 


If the rank of R is 1, then all second-order minors vanish. 


also true; for, if the second-order minors vanish, then the 
except for the trivial case when 


of Spearman’s tetrad difference 
common factor) is established 
minors in the correlational mat 


rix. 
The converse is 
rank must be 1, 
all entries are 0. The matrix interpretation 
procedure is that unit rank (i.e., a single 
by evaluating separately the second-order 
rix. 

To establish that a matrix is of any particular rank r, 


it is, of course, nec- 
essary to prove that r is tl 


he highest order of the non-vanishing minors. 
Taken literally, this requires that all minors of order higher than r must be 
shown to be 0; but for computational purposes this is probably the most 
awkward way possible, especially for the single-common-factor case. 

The tetrad difference method of examining a correlation matrix cannot 
be recommended even for the restricted single-common-factor case to which 
it is theoretically applicable. The reason is that more effective methods are 
available for ascertaining whether one common factor is sufficient to ac- 
count for the intercorrelations. If more than one factor is required, then the 
tetrad difference criterion is not applicable. Because of historical interest, 


some of the properties of the tetrad differences will be described here. * 
Because of the great amount 


of labor that is involved in the computation 
of the tetrads for a large correlation matrix, it is convenient to know how 
many tetrads must be evaluated for n tests in order to cover the whole 
table. Since a tetrad difference is the value of a second-order minor which 
does not contain diagonal terms, there are as many tetrads as there are 
second-order minors which do not involve the diagonals. Each of these 
minors is defined by two rows and two columns. The number of pairs of 
rows that can be taken is the nu 


р mber of combinations of n things taken 
two at a time, or 
(18) б = = 1) 

2 5 s 
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The number of possible pairs that can be taken from the remaining col- 
umns, since diagonal elements are excluded, then is 


(n — 2)(n — 3) 


g 


2 


(19) gr š 
Hence the total number of second-order minors in the correlation matrix, 
excluding diagonals, is 


TN n(n — 1)(n — 2)(n — 3) 
(20) ОО = 1 n 


But, since the correlational matrix is symmetrie, it follows that every one 
of these minors is duplicated by a symmetric minor of the same value. 
Hence the total number of different tetrads is 


aes, ТИ! == 1)(n — 2)(n — 3) 
(21) 103C? ^ Я 


Since every set of four variables gives three tetrads, it is possible to obtain 
the same result by considering the number of combinations of n things taken 
four at a time. Then the number of tetrads is 


(22) 30 = n(n — Dn 2)(n — 3) А 


ExaMPLE: If the number of tests is 20, then the correlation matrix con- 


tains 14,535 tetrads. 
When the computation of all these tetrads has been made, the result is 


usually that the tetrads do not vanish. The inference must then be made 
that one common factor is insufficient to account for the intercorrelations 
of the tests. If it is found that the tetrads do vanish within the sampling 
errors, then the next problem is to ascertain how much of the variance of 
each test is attributable to the single common factor. This can be done in 
terms of the correlation coefficients by Spearman’s formula (15). 
Spearman’s procedure takes into consideration that the tetrad difference 
in (17) does not quite vanish because of sampling errors in the four correla- 
tion coefficients. If a single common factor is fundamentally present and if 
the four coefficients have known standard errors, an expression for the 
standard error of p can be derived. This has been done by Wishart and by 
Holzinger.* The experimentally observed deviations of p from zero should 
not exceed those which might be expected from the standard errors of p. 
* John Wishart, “Sampling Errors in the Theory of Two Factors," British Journal of 


Psychology, XIX (1928), 180-87; Karl J. Holzinger, Statistical Résumé of the Spearman 
Two-Factor Theory (Chicago: University of Chicugo Press, 1930), pp. 6-16. 
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This is the central idea in Spearman’s single-common-factor method. The 
tetrads in a correlation matrix are first evaluated. A frequency distribution 
of these tetrad differences is then made and its standard devi 
mined. If this dispersion is of an order of magnitude compar. 
which would be expected from the known standard errors of the tetrad 
differences, then Spearman draws the legitimate conclusion that a single 
common factor is sufficient to account for the observed intercorrelations. 
Applications of formulae of type (15) give the loading of each test with the 
single common factor whose sufficiency has been established by the fact 
that the tetrads vanish within sampling errors. In order to reduce the labor 
of computing probable errors of the tetrads, Spearman and his students have 


developed several abbreviated procedures. These are limited, however, to 
the single-common-factor case. 


ation deter- 
able with that 


Factorial reduction to unit rank 


There are several types of situations which 
practical interest, in which a factor matrix or a с 
to rank 1. Consider, fi 


are of both theoretical and 


orrelation matrix is reduced 
rst, the two-column factor matrix in Table 4. If the 


Table 4 
Factor Matriz F Single-Factor Matriz 
| 
I II | 1 
1 .60 .30 1 . 6708 
2 40 20 2 .4472 
3 .80 .40 3 8944 
4 «20: 0 4 ‚2236 
ә :80 115 5 .3354 


matrix F is multiplied by its transpose Ё”, it will be seen that the columns 
in / are proportional and that it is of rank 1. The tetrads vanish, and the 


intercorrelations of the tests can be described as well by one factor, as 
shown in the single-factor matrix. 


The reason why this result is obtained is that Matrix F is of unit rank. 
It corresponds to the conceivable psychological situation in which each test 
of a battery calls for two primary 1 abilities in the same ratio, al- 
though they differ in specificity and reliability. In practice, it is possible to 
select from a lar, roups whose intercorrelations are 
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ized into a useful theorem, namely, that if two columns of any factor 
matrix are proportional, then the two columns are of unit rank, and the 
rank of the factor matrix is correspondingly reduced. It can be easily seen 
that this situation is more likely to occur in analyzing a small test battery. 
Two factors that are actually distinct may then collapse so that they appear 
in the analysis as a single factor, with consequent confusion or error of 
interpretation. Only by further investigation of the same domain would 
the error be discovered. It is quite likely that this does happen in explora- 
tory factorial investigation where the domain covered by several factors is 
not adequately covered by tests. With these limitations of method the solu- 
tion seems to be to explore rather intensively each domain with a large num- 
ber of specially constructed measurements until the factorial composition 
of each new test can be anticipated with confidence. 

Godfrey Thomson has discussed a very interesting case of reduction in 
rank in his sampling theory. Consider a factor matrix of order nXn;, in 
which each test of unit variance is defined only in terms of the number of its 
elements, namely, ту. Let there be a total of n; elements involved in the 
domain defined by the factor matrix. Then, in the row j, we have n; fac- 
tor loadings a; distributed at random, and the remaining factor loadings 
are zeros. If each test is of unit variance, we have 


Na =1; 


(23) 2 


but, since there are n; non-vanishing elements in row j, we have 


(24) n = 1, 


so that 


Let similar notation designate test k. Then the correlation rj, will be 
the cross product of the two rows j and k. The probability that there will 
be non-vanishing cell entries in the same column for both rows j and Кїз 


рур, Where 
2 ind + 
26 )j — — and p: = — 
(26) Dc 1 Te’ 


so that 


(27) рік = —— 
n 


N 
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The correlation rj, then is 

(28) Tjk = Djpiaja4n, , 

which reduces to 


(29) ns = 


but this is the well-known case of the correlation expressed in te 
mon clements. It can be expressed 
and А, where 


rms of com- 
as the product of the single factors A; 


ic NE. ne «i 
(30? A; y and Ax n; 


so that the correlation matrix is of unit rank. This result is approached as 
nı becomes large. Here we have the case in which a factor matrix with a 
large number of columns na constructed in the manner described, ap- 
proaches unit rank. It is quite another matter to decide whether this type 


of factor matrix is psychologically plausible for the description of mental 
traits and test performances, 


Table 5 
1 2 3 4 5 
1 .56 A8 
2 .56 .42 
3 .48 .42 
4 .40 135 30 
› .32 .28 .24 .20 


1.76 1.61 1.44 1.95 1.04 


Graphical analysis of tetrads 

If one is interested in 
the second-order minors 
the tetrads, 


the task of tabulating separately the values of all 


21 a correlation matrix which Spearman has called 
then some simple graphical methods might be useful. Because 
of the proportionality of the columns and rows in this case we все that, if 
the correlational matrix is of rank 1 and if any column k is plotted against 
any other column J, then the plot is linear through the origin with a slope 
which is the ratio of the single-factor loading of test k to that of test |. An 


example can be seen by Plotting any pair of columns of the correlation 
matrix of Table 5. 


Let two points on the diagram represent tests m and n. 
are radial, then 


(31) Als fi 


If the two points 


Tin Ты? 
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so that 
(32) Timin — Tintin = 0. 


If the points m and п are not radial, the proportionality of (31) does not 
obtain, and the tetrad (32) does not vanish. By this simple graphical meth- 
od one can identify the tetrads that do not vanish. 


ре 


TEST 9 


1.00 


TEST Ц 


FIGURE 2 


If the test battery as a whole cannot be described by a single common 
factor, the plot will not be linear but will scatter, as shown in Figure 2. 
This figure shows the plot of column 9 against column 1 from a correlation 
matrix in a factorial investigation by William Brown and William Stephen- 
son.* Column 9 represents a test of pattern perception, and column 1 rep- 
resents a test of inventive synonyms. Although it is evident in Figure 2 that 


* “A Test of the Theory of Two Factors,” British Journal of Psychology (Gen. Sec.), 
XXIII (1935), 352-70. * 
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a single factor is not sufficient to account for the intercorrelations and nmt 
therefore, all the tetrads do not vanish, it is still possible for smaller groups 
of tests to be of such a character that their intercorrelations can be de- 
scribed by a single common factor. Their tetrads should then vanish. The 
smallest group for which a tetrad can be written is four te 
are represented by the two columns that are plotted. Any 
on the diagram determines a tetrad. If they lie in a radial line through the 
origin, the corresponding tetrad vanishes. If they do not lie In a radial line 
through the origin, the corresponding tetrad does not vanish. 

Figure 2 gives further information about the possibility of 


hypothesis for the tests concerned. Radial lines can be dr. 
and 17, through 12 


sts. Two tests 
pair of points 


а single-factor 
awn through 8 
and 19, through 6 and 14, with small residuals. Any 
one of these three pairs can be combined with tests 1 and 9 to form vanish- 
ing tetrads, but it does not follow that all eight of the tests an be included 
in a single-factor correlation matrix. The fact that the points spread on 
different radial lines means that other tetrads which include tests from dif- 
ferent radial lines will not vanish. It should be carefully noted that, if the 
tetrads fail to vanish, we can say that one factor is not sufficient to account 
for the correlations, but it still remains a possibility that a Spearman gen- 
eral factor is present in addition to other factors, so that more than one 
factor is required for the battery of tests as a whole. We see, therefore, that 
the tetrad criterion is inadequate for disproving a Spearman general factor, 
See, for example, Figure 5b and Figure 6 of chapler ix, which illustrate 
three-dimensiona] configurations that include an orthogonal general factor, 
which we are here calling a “Spearman general factor." The tetrads do not 
vanish for such a test battery, but it contains a Spearman general factor. 
Neither do the tetrads give any information about the existence of a sec- 
ond-order general factor, to be discussed in chapler xviii. 


Spearman’s terminology 


Spearman’s hypothesis that the 
can be accounted for by 
as the theory of two factor. 
name to his theory, Spearman had in mind tw 
intellective factor and a specific or unic 
factorial investigation was dominated Ь 
intellective factor, Spearman’ 
factors, one common 
to n dimensions 


correlations between psychological tests 
a single general intellective factor has been known 


5. The general factor was denoted g. In giving this 


0 factors, namely, a general 
(ue factor in each test. As long as 
y the central theme of the general 
8 formulation could be called 
and one unique. With the ext 
and with the principal 
multiple common factors, it is с 
two factors 


a theory of two 
ension of factor analysis 
scientific interest directed to the 
to speak of Spearman’s theory of 
unique. In order to ay 


ory of a single common factor 
nultiple-f. 


onfusing 
; опе common and one 
we shall speak of Spearman’s the 
that is consistent with that of n 


oid confusion 


, a terminology 
actor analysis. 
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When Spearman found a set of tests for which the intercorrelations were 
proportional, he called the table a hierarchy. Such a table had the charac- 
teristic that the highest correlations could be placed near the upper left 
corner and the lowest correlations in the lower right corner. Much of the 
early debate about the question of whether Spearman’s single general in- 
tellective factor was adequate to account for the observed correlations be- 


tween tests turned about the statistical question as to whether the tables 
of intercorrelations exhibited so-called hierarchical order. The table of cor- 
relations was not treated as a matrix, and the question of its rank was not 
discussed in relation to this problem.” 
Before multiple-factor analysis was introduced as an explicit psychologi- 
cal method without any assumption that there should be a single dominant 
general intellective factor, there was frequent discussion of other factors 
besides the general intellective factor g. These other factors were regarded 
as secondary to the general factor g. They were described as the “disturb- 
ers” of the tetrad equation, and they were avoided by eliminating similar 
tests from the battery. For example, if two number tests were in a battery, 
their correlation would add a number factor to the common-factor vari- 
ance, and the tetrad equations were consequently not satisfied. Instead of 
adding more number tests to the battery in order to investigate number 
ability, the preferred procedure was to keep only one of the number tests in 
the battery, eliminating the others. The number factor was thereby rele- 
gated to the unique variance, and the tetrads were undisturbed. Whenever 
these additional and disturbing factors were investigated, the accepted pro- 
cedure was, first, to extract 2 general factor which dominated factorial 
Studies, Then ‘Ke residuals were investigated in groups for additional 
s treated as secondary and residual to the domi- 
s still current among 


pe of factorial analysis 
f them have not yet adopted the multiple- 


raction of multiple factors and rotation 
tructure. The newer methods leave it as 
tor is in the battery and whether it 
rder general factor. 


factors, These were alway: 
nant general factor g. This ty 
British psychologists. Most 0 
factor methods, which include ext 
Of the reference frame to а simple $ 
a question of fact whether а general fac 
IS an orthogonal general factor oF a second-o 


Special factoring methods for unit rank 


Spearman’s single-factor formula 
rst Spearman’s formula for factoring a correlation 
s tor loading of each test may be 


ommon-fae 
edure. This method is simpler than the 


We shall consider, fi 
joi of unit rank. The single-c 
etermined by a summation proc 
n Arbor, Mich., 1933), pp. v, vi, 20. 


Multiple Factors (An m 
i at the University of Chicago 


* 
L. L. Thurstone, Theory B 
Chis publication. is odt of print, but it can be obtained 
brary as microfilm document No. 1648. 
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tetrad method for a single common factor, and it gives more information 
about the variables than the tetrad differences give. 

If the correlation r can be described by a single common factor, we have 
(1) Tjk = азар. 


The sum of column / іп the correlation matrix is 


(33) У = a, S aj. 
j=l ј=1 


Since the diagonal terms аге unknown, the summation in the left member of 
(33) is unknown and hence not suitable for computing purposes. Let (r); be 
the sum of column k, omitting the unknown diagonal entry. Then 


п 
? т 7 
(34) Oh = Sn = a, 
j=1 
and hence 
п 
35 ET 2 
Pu (уь = a, а, — а. 
7=1 
Summing for all given coefficients in Jo and omitting the unknown diag- 
onals, we have 
Я n " 
зб К коң мо,» 
(36) (0), = 5 aa; AN 
kel j=l kal 
or 
й 2 
37 108 
(37) (r), Уа Р 


where (7), кж (7), denotes the sum of 
n 


all the coefficients in Ro except the 
diagonal ones. 


Each of the two terms in the right member of (37) may be expressed in 
terms of summations of known coefficients, as follows: 
(1) 


and hence 


Tk = 0j8,, 


(38) Pit 
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Summing for column А, we obtain 


3 lx K 
(39) т = Sia ; 
j=l j=1 
and, from (34), it follows that 
1 х S 
(40) als + eil = aa 
m 
or 
" e р ББ НУ 
(41) As a= 2 dj = ke 
t "E = 


Hence the first term of the right member of (37) is 


(42) beg = © + 2) + ai. 
Е=1 5 


2 
ct 


The last term of (37) may be expressed in terms of summations of coeffi- 


cients, as follows: 


(1) тк = Ajak, 


so that 
(43) rh = aja, 


and hence 


(44) 


n 
EN ox E] ^а 
(45) 25 тк = S аў = Уа. 
a = — — 
jel Feel k=1 


Summing for column k, excep 


(46) 


t for the entry in row k, we have 
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Let the sum of the squares of the known coefficients in column Ё 


be denoted 
by (7?),, so that 


(47) (2), = Уо = 


j=1 


Then (46) can be written in the form 


(7°), T S 2 
(48) wp THEA 2. 


Substituting (42) and (48) in (37), we obtain 


i : ac De 2 

(49) (т), = ‘ag FC + а a a 
or 

Ы 55. UR. (E 

(50) (7), — 2(7)„ = aj aî ° 
from which it follows that 
(51) dic (ry — (r2). 

п | Жый ^c eu тә am 


where 


(r) = the sum of the coe 


ficients in column A, omitting un- 
known diagon 


al entry , 


(r)? = the Square of (r), , 


ll 


(02), = the sum of the se 


їчагев of the known coefficients in col- 
umn k, 


(т), = the sum of 


all known coefficie 
trix. 


nts in the correlation ma- 


Formula (51) gives the single-common-factor loading of each test in a 
correlation matrix of rank one. This formula has been given, in different 
notation, by Spearman. * 

In Table 5 we have a small corre] 


factor loadings for the first test by S 


ation matrix of unit rank. T 


o find the 
pearman’s formula (51) weh 


ave 
(0): = 1.7600 , 


ll 


(?) = 8064, 


(т), = 7.1000, 


arman's formula (51) give a, = .80. By the 
-70, йз = :60, а; = 50, and a; = 40, 


р. xvi, eq. (21). 


H= 3.0976 , 
which when substituted in Spe. 
same method we get аз = 


* The Abilities of Man, Appendix, 


bo 
N 
^1 
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A logarithmic solution 
Another direct method of solving the single-factor problem is the logarith- 


mie solution.* The logarithmic formula is 


2(n — 1) M log Tjk— x tog Tjk 
D 


59 H 

(52) log Gs = 2(n x 1)(n 25 2) ’ 
where j < k. For Spearman’s formula (51) the square of each correlation 
cocfficient must be listed, and for the logarithmic formula (52) the logarithm 
of each coefficient must be listed. The logarithmic formula is not applicable 
when some coefficients are near zero, in which case the large negative 


logarithms unduly weight the determination. 


A direct summation method | 
A third direct method of det ermining the factor loadings for rank 1 can be 


derived from the graphical representation of proportional columns. If one 
> gra 


Table 6 
1 2 $ $ 2 2 
n 48 .40 32 
1 a -56 42 185 28 
3 at 42 .30 24 
4 40 :35 29 20 = 
E 7 24 
j 32 .28 ud ! 
1.4 1.04 7.0 = 
Q9 1.76 1 "d „79 5.34 (и—5) 
PRU 1.76 1 1. 1.44 7.04 (n—1)s 
p 1.76, i -500 3.750 c/ai 
1 ; LEN 25 2.9687 v/a? 
о/а 1:000 7656 * — 
d “80 70 й 


another column 1, the slope is the ratio of a; to аг. 
гп Xn, there will be (n —2) points in such a 
are unknown, being the squares of the 
be determined by the method of aver- 


column f: is plotted against 
For a correlation matrix of order 
Plot because the diagonal entries 
quantities sought. The slope can 
ages, as 


(53) a ra 


the sums of the known coefficients in columns Ё and 1. 
hown in Table 6. In the first summation row we 
e total is sr, which here denotes the sum of all 
he matrix. In the next row record values of 
row is (s, — 81), by which the summations сап 


Where s, and s; denote 
A numerical example is 8 
have the sums в, and th 
the known coefficients in t 
(в, — rj). The sum of this 


1 2 
* Por proof see Thurstone, 0P- cit., p. 62. 
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be checked. In the next row record values of (sı — 7). The sum of this 
row is (n — 1)sı, by which that summation is checked. In the next row 
record the ratios а, /а, by equation (53). In the nex 
of these ratios. 

The square of the sum of the factor loadings is equal to the sum of all 
the coefficients in the correlation matrix, including the diagonal correla- 
tions (chapter viii, equation [9]). We then have 


(54) ms | Уе]! ! 


t row record the squares 


МЕ | 
Let the sum of the factor lo 


v, which is also the sum oft 
ties. We then have 


adings be denoted cand the sum of their Squares 
he unknown diagonal coefficients or communali- 


(55) r= 8 +0 
апа 
(56) c= Ma, 
J 

so that 
(57) з фо = с 
Divide (57) by aî, then 
(58) SE oU = c 

aj ki di o 
Hence 

mÁ 

(59) a = | = 


in which all numerical values are known except ау. Applying this multiplier 
(a: = .8 by equation [51]) to the row а/а, we get the factor loadings a, 
as shown in the last row of Table 6. 


Successive a pproximations 


In addition to direct methods, there are several methods of successive 
approximation. One of these is to i 


| insert trial values in the diagonal cells 
and then determine the factor loadi 


À 2engs by the centroid method, The squares 
of the factor loadings so determined are used in the diagonal cells for a new 


trial, and the process is repeated until the factor loadings are found to any 
required degree of consistency. For smal] correlation matrices the approxi- 
mation methods require more trials than for larger correlation matrices. 
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For large tables of twenty or more variables, the first trial is usually close 
enough for practical purposes. 

When the centroid method is applied to the case of rank 1, there is no 
problem concerning the reflection of test vectors or the changing of signs, 
and the method is then reduced to its first step, namely, the summation 
for each column of the correlation matrix. It is assumed here that if there 
are any negative coefficients in the table, the corresponding columns and 
rows have been reversed in sign so as to make all the coefficients positive. 
If this cannot be done, then the correlation matrix is certainly not exactly 
of rank 1. The first-factor loadings are then determined by the summa- 


tion formula 
(60) SE уя 


which is equation (16) of chapter viii. Here the first-factor loading for test X 
is determined by rx, which is the sum of column +, including the estimated 
diagonal, and 7, which is the sum of all the coefficients in the correlation 
matrix, including the estimated diagonal values. This first step of the cen- 
troid method is identical with a formula described by Cyril Burt for a cor- 
relation hierarchy. The principal feature of the centroid method, which 
was devised for multiple-factor analysis, is the treatment of residuals by 
reflection of test vectors and corresponding sign changes in order to avoid 
the circumstance that the column sums of the residuals vanish identically. 

In using the centroid method for the case of rank 1, we must estimate 
+ the diagonal values, and consequently the solution depends on taking trial 
values, which require successive adjustment until the solution is found to 
any required degree of accuracy. The adjustments are small when large 
correlation matrices are used. This procedure is theoretically incorrect, if 
it is the intention to extract further factors from the residuals. In that case 
the first-factor determinations again become subject to adjustment with 
new communalities in the diagonal cells. 

Hotelling’s iterative method of factoring can be applied to the case of 
unit rank, but it requires several trials for the successive adjustment of 
the diagonals which become communalities when the correct solution has 
been obtained. This solution has the characteristic, when applied succes- 
sively to a case of unit rank, that the resulting factor loadings minimize 
the squares of the residuals of the side correlations. The residuals for the 
diagonal entries are zero when the correct values have been found. 


Limitations of single-factor formulae for multiple-factor analysis 

In dealing with actual data it is always a question of fact whether any 
given table of intercorrelations is of rank 1. When a correlation matrix is 
to be analyzed with the assumption that it is of unit rank, it is, of course, 
implied that only one factor needs to be extracted. The values inserted in 
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the diagonal cells are then the squares of the factor loadings. If the diagonal 
values are known, then the factor loadings are also know 
case of rank 1, it is possible to solve directly for the dia 
are the squares of the desired factor loadings. Sever: 
been described here, and others could be devised. In 
are not of great importance because it is rare th 
a rank as low as 1. 

The student should be cautioned that if he solves for the first-factor 
loadings by any of the single-factor formul 


n. In the special 
gonal values, which 
al such methods have 
practice such methods 
at a correlation matrix has 


ae, and then attempts to extract 


Table 7 


Correlation Matrix 


1 .50 E .30 .21 
2 .50 .58 .44 .84 
3 .41 .58 :54 BT 
4 .30 44 .54 .62 
5 21 .84 ‚57 .62 
За p 2.10 — 1.900 1.74 9.02=(r), 
.5522 ^: 1.1210  .9596 .8690 
2.0164 3.4 4.4100 3.6100 3.0276 
.2369 -6824 .5077 3806 
49 .83 Eu -62 Ву equation (51) 
Factor Matriz First-Factor Residuals 

Eu. = =———— 

ЕКТ М 1 2 3 4 5 
вне ale a AE Et oe 
2 | .2 8 .68 2 .16 00  —.06  —.09 
3 [| 59:6: 461 3 .00 .00 —.05 .06 
x 5B adi 55 4 =  —.06 ~.05 18 
9.199. 2 85 5 —.09 —09 .06 18 


additional factors in the same manner fro 
be theoretically incorrect, This f. 
cance, and it may be worth w 
nature of this error. 

In Table 7 we have a factor matrix for five v. 
The communalities are listed in the 
"re 3 shows a plot of the two. 


m the residuals, the solution will 
act is of considerable theoretical signifi- 
hile to give a numerical example to show the 


ariables and two factors. 


last column of the factor matrix, Fig- 


-dimensional configuration. The correspond- 


ing correlation matrix is also shown in Table7. Now Suppose that this corre- 
lation matrix is analyzed by one of the single-faetor formulae. We have 
taken Spearman’s formula (51) for this example. The first-f, 


Я actor loadings 
are shown in the row ar, and the squares of these log 


ings are also listed. 
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By using these factor loadings we can compute a table of first-factor residu- 
als, as shown at the bottom of Table 7. 

The correlation matrix was constructed so that it is of rank 2. After ex- 
tracting one factor, the first-factor residuals should be of rank 1, but inspec- 
tion of the residuals shows that such is not the case. Furthermore, the 
square of the first-factor loading of the third variable is higher than the 
communality of that variable for two factors, as shown in the given factor 
matrix. The square of the factor loading is .68, as determined by the single- 
factor formula, whereas the communality is only .61. Therefore, there exists 
no axis in the configuration of Figure 3 on which the five tests have the 
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FIGURE 3 


projections given by the single-factor formula. These factor loadings are 
therefore impossible if the rank is assumed to be higher than one. The only 
way in which the single-factor formulae can be justified is to regard them as 
single-factor descriptions of the correlation matrix, in which сазе the first- 
factor residuals are regarded as variable errors. If the attempt is made to 
analyze the correlations as due to more than one factor, then the commu- 
nalities must be adjusted to fit the higher rank. Any valid factoring for- 
mula should give a set of factor loadings which are the projections of the test 
vectors on an axis in the configuration. The same idea сап also be expressed 
by saying that the factor loadings should be linear combinations of the col- 
umns of correlation coefficients. The single-factor formulae give correct 
factor loadings, or approximations to them, when the correlation matrix is 
regarded as of unit rank or as an approximation to that rank. 


CHAPTER XIII 
THE UNKNOWN COMMUNALITIES 
Nature of the problem 

The communality of a test is its common-factor variance. The comple- 
ment of this variance is the uniqueness of the test, as previously defined. 
The term “communality”* was adopted to distinguish the common-factor 
variance from the reliability coefficients, which are conventionally recorded 
in the diagonal cells of a correlation matrix. The notation Аў was adopted 
for the communality so that h; was the length of the test vector j in the 
geometrical representation of the common-factor space. The communality 
is, then, the square of the length of the test vector, and it is the scalar 
product of test J with itself, This interpretation of the diagonal makes it 
consistent with the side correlations, so that the whole correlation matrix 
represents scalar products in the common-factor space. To estimate the 
communalities is, in fact, to make a cleavage between the common-factor 
variance and the unique variance of the test battery. If this is done badly, 
then the identification of the common factors and their relations is ob- 
Scured in the resulting factor matrix. 

No matter What values 
theorem that the number 
intercorrelations of n lests is 
If unity is written in the diagonal c 
“complete correlation matrix №; 


are written in the diagonal cells, we call it 
and the theorem then states that the 
lent common factors in a battery of n tests is the 
rank of their reduced correlation matriz R. In making a factor analysis one 


f common factors is smaller than the 
number of tests, so that r < n. 


Since the diagonals are unknow 
the communalities so that they 
correlations, subject to the restric 
tive numbers between 0 and +1. If we tak 
too high, then we shall increase the r 
due to the diagonals. Factoring such а m 
unique variance into the factor matrix, w 


n, we have the problem of determining 


* D. L Thurstone, Theory of Multi 


ple Factors (Chicago, 1933), p. 8. 
į Ibid., p. 20. 
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sult. If we take the estimated communalities too low, then we destroy the 
Gramian properties of the matrix, so that it is по longer a correlation matrix. 
If we take the communalities at their correct values, then the rank of the 
side correlations is maintained by the communalities, subject to the restric- 
tion that they be positive numbers between 0 and +1. 

The relations between the experimentally given side correlations and the 
unknown communalities ean also be stated in the theorem that the smallest 
number of independent factors that will account for the intercorrelations of n 
tests is the minimum rank of the correlational matrix with the diagonal entries 
treated as unknown positive values between 0 and +1.* This theorem has been 
misinterpreted to mean that the rank is itself a variable, which can be given 
different values, and that the minimum rank should be chosen for which the 
communalities take possible values. The theorem is operationally true as 
far as computational procedures are concerned, but the essential fact about 
the communalities is that they are really determined by the side correla- 
tions, which are given. The problem is to determine the communalities 
which are implied by the given side correlations. When found, they do give 
the minimum rank for which the communalities have pos ible values. А 

In dealing with experimentally determined correlation coefficients, it 
must be remembered that the sampling errors of the coefficients are fortui- 
tous and that, consequently, the rank of the correlation mat Bom experi- 
mental values is, in general, equal to its order, namely, n. The Soe 
problem of analyzing а set of experimentally given correlation сше їз, 
then, to find another correlation matrix of minimum rank thats discrepan- 
cies from the experimentally given coefficients um small eres i ba ig- 
nored. We may summarize the multiple-factor probier as ree qu 
the experimental data we obtain the side correlations, and the d iagonal ¢ Hs 
are left blank. A matrix of minimum rank is to be obtained [ve е 
only slightly from the given side correlations. тыаны E S е 
in so as to be consistent with the side correlations. We then have the re- 
duced correlation matrix which is to be factored. In Dake: у 
tational procedure does not follow this route, which is merely a statement 
of the theoretical problem. 


* ls 

i a A p een ‚ Thomson has discussed this aspect of the factor problem as if the 
objec ыс o quis the number of common factors (The Factorial Analysis of Hu- 
s ect pors 10 A ым shton Mifflin Co., 1939], chap. viii). Operationally, and as re- 
nan Ability (Boston: 8 ay that when the communalities have been 


it is correct to 5 
ards c ions hods, it is correct , al 
a pe не rank is a minimum. If the communalities were taken 
ritten i iagonal с! E re ^ ki Р oj (à 
sm: ee А thg Se odd pid If the communalities were taken larger, then the rank of 
smaller, ne rank Wi г? 


the matri 1d also rise. This problem is resolved if we realize that the communalities 

1€ matrix wi also rise. : 2 it ig Бо ааа scm 

are ee im Š ed by the given side correlations, and it is ier ely a statistical prob- 

len E d ma americ values or approximations to them. The theorem is correct 

in ü tbe ne malities maintain the rank of the side correlations, and this is the mini- 
а! 1e communi 

mum possible rank. 
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High and low diagonal values 
Among students of factor analysis it is universal practice to leave the di- 
agonal elements as unknown when dealing with the case of unit rank. This 
is Spearman’s case of the central intellective factor. The tetrad difference 
equation is so written as to avoid the unknown diagonal elements, and 
Spearman’s single-factor formula is also written 80 as to avoid the unknown 
diagonals. When the single-factor saturations or loadings have been de- 
termined, the communalities become the squares of these loadings. The cor- 
relation matrix is, then, of unit rank, with all cell entries determined. Some 
students of the factor problem deviate from this logic as soon as they deal 
with higher ranks. Then they insert unity in each diagonal cell, regardless 
of the rank of the side correlations. This leads to an erroneous factorial 
solution if the purpose is to analyze the factorial composition of the tests as 
given by the correlation coefficients. 

The question as to whether unity or the communalities are to be recorded 
in the diagonal cells is determined by the purpose of the factor analysis. If 
the purpose is to reproduce the original test scores as closely as possible by 
means of any specified number of factors, then unity should be written in 
the diagonal cells. This is true even for Spearman’s hierarchy if the purpose 
is to reproduce the actual test scores. If the purpose of the factor analysis 
is to analyze the common factors that produced the intercorrelations, then 
the communalities should be written in the diagonals. The objective is then 
to reproduce the intercorrelations as closely as possible. When unity is writ- 
ten in the diagonals, the original test scores can be reproduced with small 
residuals. When communalities are written in the diagonals, the intercorre- 
lations can be reproduced with small residuals, but the original test scores 
will leave larger residuals because the unique factor in each test is unac- 
counted for, The problem of analyzing the common factors that are repre- 
sented by the Intercorrelations is likely to be the more significant for scien- 
tific work, but there are legitimate problems that call for unity in the di- 
agonals. The two objectives—to account for the test scores and to account 
for the intercorrelat ions—merge in the same correl 
communalities are unity. They differ even in this case, however, in the ref- 
erence frame that is chosen for final description of the variables. If the test 
scores are to be reproduced аз accurately as possible with a small number 
of factors, the principal axes of the test configuration is the indicated ref- 
erence frame. If the intercorrelations are to be reproduced as accurately 
as possible with asmall number of factors, then the reference frame is chosen 
by the principles of simple structure if it can be found in the configuration. 
If it is not found, then the reference axes may be placed according to clusters 
or in regions of the configuration which can be identified with any con- 
venient descriptive names, even though the axes so located do not repre- 
sent well-defined functional unities that are indicated when a simple strue- 


ation matrix when the 
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ture can be discovered in the configuration.* The principal axes сап also 
be used in this case if there is no intention to interpret the factors. 

In Table 1 we have a small numerical example that has been prepared to 


show the effect of taking communalities that are too high or too low. In the 


first section we have a correlation matrix for three tests. Unity has been 


Table 1 
1) d= Correlation Matrix Factor Matrix 
i 2 3 I п ш 
1 L00 -— .48 .56 1 .8884 .2797 —.4677 
s 10 -— 42 ә | —17809 6247 .0000 
: 56 = 42 100 3 181388 849 14677 
2) d= 
1 2 3 З 
1 j4 = .48 .56 1 .80 
Be i = d 36 = 42 2 | 90 
8 56 = .42 .49 3 70 
_ р الي‎ 
3) 1»d»4 
Я а 3 I п ш 
e 56 1 .8009  .1240 —.2078 
bii 10 2$ 10 ә | —644 2911 “0000 
E Se шә 00 3 7273 1161 12078 
4) а<м 
j 3 3 I п III 
= - 3 56 1 .8031 .10702 =. 18314 
1 60 = 48 = u2 2 .9591i .0000 
2] = 48 o8 “40 3 1i .183H 


sponding centroid faetor matrix is also 

as many factors as there are tests, the test 
exactly, as well as the intercorrelations. In this 
of condensation or simplification, 
e number of tests. We might, then, 


Written in the diagonal, and the corre 


shown. Since we have here 
scores should be reproduced 
case we have gained nothing by way 
since the number of factors is equal to th 

* An example of a factorial analysis with a set of cers атан axes that are 
not primary factors is given in а paper by the write = : T e m Study of Vo- 
cational Interests,” Personnel Journal, X (1931), 198-205. The configuration was not 


analyzed for simple structure- 
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just as well use the original test scores as they stand, without any factor 
analysis. However, if it is desired to represent the test scores with fewer 
factors than tests, the factor matrix here shown could be rotated to the prin- 
cipal axes, and the major principal axis could be used as the best single- 
factor representation of the scores. In that case the intercorrelations would 
not be well reproduced. 

In the second correlation matrix we hay 
we have here the communalities th 
corresponding factor m 


e the same intercorrelations, but 
at reduce the matrix to unit rank. The 
atrix shows one factor which reproduces the inter- 
correlations exactly. The test Scores would not be well reproduced by this 
factor matrix. In fact, no single-column factor matrix can be written that 
would reproduce the test Scores, since each test has a unique factor, 

In the third correlation matrix we have the same intercorrel 
here we have written diagonal values that are less th 


than the communalities, The corresponding factor matrix is also shown, 
and it will be seen that here again it shows three factors, 

Finally, in the fourth correlation matrix we have the same intercorrela- 
tions, but we have written diagonal values that are less than the commu- 
nalities. Here the Gramian properties of the matrix have been destroyed. 
The principal minors are no longer positive or zero, and consequently this 
cannot be a correlation matrix. It can be factored by the centroid method, 
however, and the resulting factor matrix is shown. It is seen that the factor 
loadings contain imaginaries, 

The examples of Table 1 serve to illustrate the principle that the commu- 
nalities should be so chosen that they are consistent with the rank that is 
determined by the intercorrelations. When the di 
alities, the rank 0 
order. The same result h 
the communalities, in w 


The rank of the matrix 


ations, but 
an unity but greater 


agonal elements exceed 
f the whole matrix is, in gencral, equal to its 
appens when the diagonal elements are lower than 
hich case we have imaginaries in the factor matrix. 
А does not seem to be in any sense a continuous func- 
tion of the communalities, On the contrary, the communalities are, in gen- 


eral, a singular set of diagonal values that are determined by the side corre- 
lations. In general, the rank of ac 


orrelation matrix is either n or r, but 
special cases could be constructed in 


0 which diagonal values could be chosen 
to make the rank take intermediate values between r and n. This could be 


done in fictitious examples by using the communalities in some of the diag- 
onal cells and higher values in the rest of the diagonal cells. It is evident 


that the diagonal values cannot be chosen so as to reduce the rank below 
that of the given side correlations, 


It should be recalled that the unique variance of each test is determined 
entirely by the other tests with which it is associated in the test battery 
that is being analyzed, Except for the variable errors in the unreliability of 
a test, it can be associated with other tests that involve the same factors so 


the commun 
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as to bring its communality up to the reliability of the test. This is, in fact, 
the challenge of factor analysis, namely, to formulate hypotheses about the 
that produce the intercorrelations and then to combine 
ance with other tests in such a manner that 
In that case the 


nature of the factors 


a test of unknown unique vari 
the common-factor variance becomes completely known. 
communality is equal to the reliability of the test.* 


Minimum rank and minimum trace 
The trace of a correlation matrix is defined 
ties, and it may be denoted ё. Then 


as the sum of the communali- 


SUE 


j 


(1) 


The problem has been raised by Godfrey Thomson as to whether the trace 
of a correlation matrix is a minimum when the diagonal elements have been 
so chosen as to minimize the rank. The question then is whether or not 
minimum rank and minimum trace coincide in the same set of diagonal 
values, The problem was investigated by Walter Ledermann, Î who finds 
that, while the attainment of minimum rank by the choice of the proper set 
of diagonal elements gives a small value for the trace, this is not necessarily 
the minimum possible value for the trace for which the matrix remains 


Gramian. ЕД oe, 

For the case of unit rank, the trace of the communalities is a minimum 
^as ank, 

mmunalities, Professor Kelley considers a 

involve а certain musical capacity not pres- 
A т alities of the music tests include their variance 

ent in the other eight tests. The communalities 0! 4 H h 

in the music fact Б since it is common to two tests in the battery. Now, if one of the 

0 Плата ical test will have a reduced com- 


i ү, ther musi 
Musical tests is dropped from the battery, the о ы 5 
munality An pene aa js due to the music factor, which then becomes a specific fac- 


acity i ic reflected in part in the one 

tor and i zer 8 с 1 factor. “Capacity in music re! i 

fest ia ey сек A cit and according to Thurstone not worthy of retention in 

e T Т Crean e Kelley, IMMER ERE of Mental Life (Cambridge: 

Нагу, күш rius 1935] not a question of whether the 
arvar niversity Press, 1799)» 


р. 56). It is, of course, 
Specific musical ability is “worthy” of retention. The question is how to disentangle the 
nusica 1 ү 3 5 А i үн | 
music factor from other specific factors т the same test and from the error variance in 
that test т sie ation of the specifie music factor from the other specific factors in 
est. The separa iege 
the remaining musical test js not solved by writing А 


in the diagonal cells. But the 
problem сап be solved by using ап augmented test battery, writing the communalities 
ч an b ү у : 
in the diagonal, and rotating the axes in the comme 


-factor space. Professor Kelley 
does not approve of this procedure, but he has not offered any other solution for the 
problem. : 

+ “On a Problem concerning Matrices with Variable Diagonal Elements," Proceeding 
9f the Royal Society of Edinburgh, LX, Part I (1939-40), 1 17. 


to the use of co! 


* In stating his objections i 
s ts, two of which 


hypothetical battery of ten tes 
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except when one of the loadings is larger than the sum of the others. If we 
let hı be the largest saturation, we have, for the exceptional case, 


(2) hi Mu. 


j=2 


The exceptional case is obtained by taking 


another set of values u; so that 
the new diagonal values become 


(3) dj = MÀ + иу. 
A unique set of values of u, is obtained from the relations 
(4) ч = —h(h|— ha — hg —--- hn); 


(5) ui = Ahi — hs — =), (j = 2, Sr Se ANE 


The trace for the new diagonal values d; is then smaller than the trace for 
the unit rank diagonals h} by an increment e, which is 


(6) e= Su = (ir> Ne hym nao ЛЫ 
4 


but the rank of the matrix with diagonals d; is (n — 1). The matrix is still 


Gramian. 
Table 2 
Correlation Matrix Factor Matrix 
1 2 3 4 5 I 
ا‎ c 

1 .2250 .1800 .0900 1 .90 
2 .225) .0500 .0250 2 „25 
3 - 1800 . 0500 .0200 3 .20 
4 .1350 — .0375  .0300 .0150 4 15 
5 .0900 | .0250 .0200 5 .10 
he . 8100 ‚0625 .0400 .0100 — a =: 

trace = . 9450 rank =1 
Ln —.1800 .0500 .0400 .0300 .0200 
dj . 6300 - 1125 .0800 -0525 .0300 

trace = . 9050 rank = (n—1) =4 


In Table 2 we have a numerica 
5X5 is of unit rank w 
ing single-column f. 


И example. The correlation matrix of order 
hen the communalities h} are used. The correspond- 
actor matrix is also shown in the table. The trace for 
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h} is 9450, and the rank is 1. The corrections u; were computed by (4) and 
(5), producing the new diagonals d; as shown. The new trace is lower, name- 
ly, .9050, and the decrement is (.20)?=.04 by equation (6) with rank 
(n—1). The same author investigated the minimum trace for higher ranks 
with the same conclusion, namely, that the minimum trace for which the 
matrix remains positive-definite is not necessarily that which gives mini- 
mum rank. 

On the basis of these findings we can write the useful theorems that the 
communalities are a set of diagonal values which retain the rank determined by 
the side correlations. The communalities give the matrix its minimum rank, 
namely, the rank of the side correlations. The minimum trace does not neces- 
sarily give the minimum rank even when the diagonals are so chosen that the 
matrix remains Gramian. In practice, it may be expected that the matrix 
will no longer be positive-definite when the diagonals are chosen lower than 


the communalities. 


The Heywood case 

A special correlation matrix that has attracted the attention of some stu- 
dents of factor analysis is that in which the rank is 1 (Spearman’s hierarchy) 
but in which one of the diagonal elements must be greater than unity in 
order to preserve the low rank. A correlation matrix of rank 1 in which a 
diagonal entry exceeds unity is called a “Heywood case."* This curious 
type of matrix will be discussed here with the example used by Godfrey 
Thomson in his analysis of this interesting case. f 

The correlation matrix of Table 3 is of order 5X 
far as this can be determined by the proportionality of the columns of side 
correlations. If we determine the single-factor loadings that will produce 
the correlations, we have the single-column factor matrix as shown in the 
table. Since one of these loadings is greater than unity, we have here a Hey- 
wood case. Instead of worrying about this circumstance as if it were some 
insuperable obstacle in factor analysis, it would seem better to regard this 
finding as showing merely that the first test does not conform to unit rank, 
which has been imposed on this correlation matrix. If we confine our atten- 
tion to the other four tests, we have a clear case of unit rank, but not when 


and it is of rank 1, as 


all five of the tests are considered together. 

In the same table we have exhibited one of an infinite number of con- 
sistent factor matrices that could be written to cover this case. We have 
shown, with a unique variance for each test. 


here a common factor, as a | 
These are arbitrarily written so that the total variance is unity for each of 
2-5, but consistent factor matrices could be written for this 


the four tests 2-4 
* H. B. Heywood, “On Finite Sequences of Real Numbers,” Proceedings of the Royal 
Society, A, CXXXIV (1931), 486-501. 
f Op. cit., p. 231. 
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case in many other ways. Test 1 is shown here as a complex test. If this 
consistent five-column factor matrix is multiplied by its transpose, then 
by the fundamental factor theorem, FF'=R, the given correlation matrix 
is reproduced. The communality of test 1 is now -9802, which is less than 
unity, and the mystery of the Heywood case has been resolved. It could 
have been done in many other ways that imply more th 
the complexity of the offending test 1. It is advisable to restate here the con- 
dition under which a correlation matrix can be accounted for by asingle com- 
mon factor. A correlation matrix can be described by a single common factor if 
it is of unit rank (proportional col umns) and if the diagonals are positive numbers 
between 0 and +1. If these conditions are not satisfied, then more than one 


an one factor in 


Table 3 


Correlation Matriz Factor Matrix 


1 2 3 4 5 I 
1 .945 .840  .735  .630 1 1.05 
2 | .945 .720 .630  .540 2 .90 
3 | .840 .720 .560 1480 3 .80 
4 | .735 .630 .560 .420 4 .70 
9 | .630 .540 480  .420 5 .60 
Consistent Factor Matrix 
— — 
| I п ш IV v In 

J | .90 .3097 .2000 .1470 1125.  . 9802 

2 | .90 .4359 1.0000 

3 -80 6000 1.0000 

4 -70 E 1.0000 

9 -60 -8000 1.0000 


common factor is required to describe the correlations. Mathematically, a 
symmetric square matrix with proportional columns and positive diagonals 
can be described as the product of a single-column matrix and its transpose, 
but the product cannot be a correlation matrix if any of the diagonals are 
negative or if they exceed unity. 

The treatment of analogous Heywood с 
is similar to that for unit rank. In the е 
ance of a diagonal value greater th 


the factor matrix is higher than th 


ases in multiple-factor analysis 
ase of multiple factors the арре 
an unity means merely that the r 
at which produced the Heywood ¢ 


ar- 
ank of 
ase. * 

* Godfrey Thomson (bid., p. 234) discusses the analysis of Spearm: 
a large number of tests, with the conclusion that a Heywood с: 
When the number of tests is the same as the number of persc 
correlation matrix vanishes, Thomson concludes th 
cannot vanish with unity in the diagonals except by 


an’s hierarchy for 
ase would necessarily arise. 
ns, the determinant of the 
at the determinant of a hierarchy 
introducing a Heywood case, і.е., а 
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The number of independent factors 

In the analysis of experimental data in relation to some hypothesis, one 
always considers, either explicitly or implicitly, whether or not the experi- 
mental observations are adequate to sustain the hypothesis. Attention usu- 
ally centers on the discrepancies between the fallible experimental observa- 
tions and the proposed theoretical relations. This is the principal domain 
of statistical method. But even before such analysis can be undertaken, one 
must satisfy an even more fundamental requirement. The number of ex- 
perimentally independent observations or readings must exceed the num- 
ber of independent parameters in the hypothesis in order for the demonstra- 
tion to be determinate and convincing. This principle is universal, and it is 
so fundamental that every scientist takes it for granted intuitively even 
when he does not formulate it explicitly. 

We shall consider a simple example of the principle. Suppose that the hy- 
pothesis states that two variables are proportional. Writing this hypothesis 
explicitly, we have y = kx for all observations. There is only one parameter 
in this hypothesis of proportionality. The hypothesis would be represented 
by a straight line through the origin, and all observations should lie on that 
line. Suppose that we take only one observation, which consists of an ob- 
served numerical value for c and the associated numerical value for y. 
Plotting the observation on cross section paper, we have only one point. 
No one would accept such a demonstration, because one point on the graph 
does not demonstrate that the two variables are proportional for all pos- 
sible values of x and corresponding values of y. The hypothesis involves 
only one parameter, and we have only one observation. We must have 
more observations than there are independent parameters to be determined. 
Two observations which fall on a line through the origin would look prom- 
ising, but they would not be convincing. Five or six БАЕ TALONS or twenty 
or thirty eld be convincing. We demand intuitively in any brin 
demonstration that the number of experimental observations exceed the 

the hypothesis that is to be sustained. 


number of independent parameters in 2 A 3 

This is, in one e e un opposite of the mathematical requirement in solv- 
51 one sensi к 8 ч E 

ing simultaneous equations. If we have more simultaneous equations than 


unknowns, we are afraid of inconsistencies. In scientific work the parane 
ters must ‘he many times overdetermined, with small experimental incon- 


i 7 sis as plausible. 
Sistencies, before we accept а hypothesis as pla 


he theorem that if a correlation matrix 
nk, then all side correlations are either +1 or —1. 
o be expected. Then all the tests are per- 
d there is no Heywood case. If the side 
ded in the diagonal cells, then the de- 
k is then higher than 1 and there is 


ater than unity. We have tk 


f unit rar 
в exactly as is t 


diagonal element gre: 
With unity in the diagonals is 0 
The determinant then vanishe 
fect measures of the single common factor, an : 
correlations are less than unity and if unity is recor 
terminant still vanishes when ? N, but the ran 


no Heywood case. 


292 MULTIPLE-FACTOR ANALYSIS 


In a large and more complex problem one might forget this principle in 
the attempt to determine more parameters than there are experimentally 
independent observations. In factor analysis it is useful to know how many 
independent factors can be determined from any experimentally given cor- 
relation matrix. In order to find this relation we consider the number of 


experimentally independent correlation coefficients for a battery of n tests 


and the number of linearly independent factor loadings in a factor matrix 
for n tests and r factors, 


The number of intercorrelations in the correl 


ation matrix of Table 4 is 


n(n — 1) 
> , 


since we regard the diagonal elements as unknown. In considering the num- 
ber of linearly independent cell entries in the factor matrix, we must take 


Table 4* 
Correlation Matrix 


Factor Matrix 
n tests r factors 
n tests 1 2" ¥ ou 5 $ 4 8 n tests Ir ЧЕ dr av 
1 1 0 0 
2 Tio 2 Qi. D 
3 Tis Tag 3 as 0 
4 Ти Tu Tu 4 аз ац 
2 Tis 7% 7 Tg 5 аз аң 
б Tie Т 7% тє Fig 6 аз ад 
7 tor fa Te та fu re 7 аз аң 
8 Ti: Tw fus Те fj та Fi 8 Od. Uu 
n(n—1 Н з 
та) coefficients | nr parameters 
* This example shows that 


at five common factors cannot be determined from the intercorrelations of eight tests, 


into consideration that the с 


onfiguration of test vectors ean be so rotated 
that a cert 


ain number of zero entries can always be introduced. These are 
indicated in the factor matrix of Table 4. The first axis сап be placed so as 
to be collinear with the first test vector, The second orthogonal axis can 


be placed in the plane of the first two test vectors. The third axis can be 
placed orthogonal to the first two tes 


А t vectors in the space spanned by the 
first three test vectors. The fourth axis can be placed orthogonal to the first 
three test vectors in the four-dimensional space that is spanned by the first 
four test Vectors, and so on. This will introduce a number of cell entries 
that are identically zero. The number of such entries is 37(r—1), and hence 


the number of linearly independent factor loadings in F is 
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In order that there may be a unique configuration of test vectors for the 
correlation matrix, the number of experimentally independent correlation 
coefficients in 2 must equal or exceed the number of linearly independent 


parameters in F. Hence 
Ө r(r —1) a(n — 1) 
U) gre————— چ‎ 
2 EY 2 
alue of r for a given value of n is represented 


The re А 
1e condition for a maximum v 
he inequality. The condition then 


by substituting an equality sign for t 


becomes 

(8) ms r(r 5 1) e n(n = 1) 

or 

(9) on т 1) RYE I) = 0. 


we have the following theorem: Jn order that 
known diagonals for n tests and r common 
figuration, it is necessary that* 


Solving the quadratic 1n 7, 
the correlational matrix В with un 
factors shall represent а unique con, 


on +1) — Vn +1 
(10) pee 


The suppression of the po fore the radical in (10) is justified by 
the postulate that < n- When the equality sign is used in (10), the value 
of r becomes integral for certain values of n. Then the number of independ- 
ent parameters of F is exactly equal to the number of experimentally inde- 
pendent coefficients in A. Such is the case when n=6 and r=3. We see 

at least three tests are required to determine one 
at least four tests are required in order that 
termined. This is the situation in Spear- 
which involves four tests for the deter- 
At least five tests are required to de- 


sitive sign be 


also, for example, that 
common factor and that 
one common factor shall be overde 
man’s tetrad difference equation, 
mination of a single common factor. 


termine two common factors. 
(9) is symmet 
(10), so that 


Since equation rie in n and r, n can be expressed explicitly 
in terms of r by analogy from 
(11) п > 3 д 
aber of tests required for the determi- 
»ws, for example, that there must be 
on faetors. 


Bae Кс 
This relation shows the minimum nun 


nation of r factors. Equation (11) sho 
at least cight tests in order to determine four comm. 
also E. B- Wilson, 
of General Psychology, 


«“povyiew of Crossroads in the Mind of 3 ч 
* Ist ed., р. 76; see Revit (1929), 158. TAURUM 


T. L. Kelley]," Journal 
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An actual count of the correlation coefficients in Table 4 shows 28, where- 
as the count of independent parameters in the factor matrix is 30. Hence 
the five indicated factors cannot be determined from the correlations of eight 
tests. Four factors can be determined, since 28 > 26; but this calculation 
assumes that the correlation coefficients are infallible. With actual data one 
would not ordinarily attempt to find more than two or, at most, three fac- 
tors for eight tests. 

It is useful to have a table to show the smallest number of tests that will 
just determine a given number of factors or the largest number of factors 
that can be determined by a given number of tests. This information is 
summarized for ten factors in Table 5. This table must not be used to de- 


Table 5 
No. of No. of | No. of No. of 
Factors Tests | Factors Tests 
1 3* 6 10* 
2 5 T 12 
3 6* 8 13 
4 8 9 14 
5 9 10 15* 


. *The asterisks refer to integral values of 
both r and » in (11). 


termine the number of factors to extract from experimentally given data. 


A battery of fifteen tests cannot be expected to determine ten factors with 
any degree of stability. In 


tifically significant and с 
times greater than those which 
1 í can be determined from the intercorrela- 
tions of a given number of tests, as far as the count of independent param- 


onsider the degree of overdetermination 
fic acceptance of a factorial interpretation 


Methods of estimating the communalities 
Before the numerical work of factoring is beg 

mate the communalities. The estimation of ¢ 

sidered under three general heads, namel 


un, it is necessary to esti- 
ommunalities will be соп- 
у, (1) when exact, or nearly exact, 
actoring is to be done with adjust- 
factor has been extracted; and (3) 
djustment in the diagonal cells after 
rd case implies any factoring method 
als after each factor has been com- 
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The first case is that in which it is desired to estimate the communalities 
very closely. For large correlation matrices it is seldom necessary to insist 
on close estimates of the communalities. The reason is that in this case the 
factorial results are not seriously affected by coarse initial estimates for the 
diagonal cells. It is another matter when the correlation matrix is small, 
say, of order 8 or 10 or less. For small correlation matrices the factorial re- 
sults сап be markedly affected by wrong estimates for the communalities. 

The best procedure for small correlation matrices is probably to do the 
factoring with such estimates as can be made. The resulting factor matrix 
will then give a new set of communalities, which can be reinserted in the 
original correlation matrix. The factoring can then be repeated with the 
new communality estimates. But in the subsequent trials one does not 
make any adjustment in the diagonal cells after each factor. After a few 
trials it will be found that the communalities determined by the factor 
matrix are practically the same as those with which the factoring was 
started. These values may then be accepted as close estimates of the com- 
munalities. It is a fortunate circumstance that the necessity for repeating 
the factoring in order to get stable factorial results and stable communali- 
ties occurs with only small correlation matrices for which the computational 
labor is not prohibitive. For large correlation matrices this repetition of the 
factoring is seldom if ever necessary. 

In dealing with experimental data it must be recalled that the rank of the 
correlation matrix is, in general, equal to its order. It is then a question of 
statistical judgment when the residuals are small enough so that they can be 
ignored. If the factoring is continued without adjustment in the diagonal 
cells after each factor has been determined, the residuals will be identically 
zero after n factors have been extracted. This is true for the various cen- 
troid methods and for other methods of factoring. In practice the com- 
puter stops the process before such a result has been obtained. He stops 
extracting further factors when he judges the residuals to be sufficiently 
small. At this stage it is evident that each additional factor contributes 
o the total variance of each test in the common factors. Since 
each factor contributes а small increment to the communality, it is evident 
that the estimated communality must be judged in relation to the number 
of factors that are extracted. It follows also that the true communalities 
can be determined only by extracting (n—1) factors with m tests in the 
battery. It would usually be a waste of time to carry the work to this 
extreme, and consequently the true communalities are not ordinarily de- 
termined for experimental data. In determining the best-fitting communali- 
ties by repeating the factoring several times, we must remember that each 
trial should be taken to the same number of factors. The revised estimate 
of the communalities should then. be stated for the number of factors that 
were determined. When it is desirable to obtain a very close estimate of 


very little t 
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the communalities for a specified number of factors, the ideal procedure is 
to obtain the principal-axes solution to the specified number of factors and to 
repeat this process with adjusted communalities, as determined in each trial. 

Our second case is that in which a factoring method is used which gives a 
set of residuals after each factor. There is then an opportunity after each 
factor to adjust the diagonal cells. The simplest method is that of recording 
in the diagonals the highest. coefficient in the column. This method has 
been used in a large number of studies with satisfactory results. 

Our third case is that in which the diagonals are not adjusted after each 
factor. This is necessarily also the case when the factoring is done with- 
out computing residuals after each factor. In this case it is necessary to 
make a fairly good estimate of the communalities before the factoring is 
begun. If in this method of factoring it should be discovered that a large 
error has been made in estimating the diagonals, it might be necessary to 
repeat the factoring in order to correct the error. This will be true espe- 
cially for small correlation matrices. 

A number of principles and computing formulae will be considered by 
which the communalities may be estimated. Most of these formulae are 
approximate, 

1. Expansion of a minor of order (r+1) 

If the correlation coefficients are infallible and if the rank of the correla- 

tion matrix is assumed to be r, then the communalities can be computed on 


Table 6 
Factor Matrix 


Correlation Matrix of Rank 2 


ЕЕ Eee IM ANM ITE NECI E 
1 28. 0 ; 56 16 72. .64 40 
2| 7 4 2 56 38 7 .72 „08 
3 2 .6 3 16 .38 .24  .40  .52 
4 0 -7 4 | .24  .49 .94  .52 .64 
к Pep. S || ao. 07 34 A6 0.52 
6 8 E: 6 .64 .42 .52 .76 .68 
7 .5 «й 7 | .40 .63 .64 .52 .68 
8 3 8 8 .24 53 .65 .35 .56 ‚71 
‘True com- j 
munali- | 
Da | A 
ties -64 65 -40 .58 82 .80 74 73 


the basis of this assumption and tested for consistence 
pute the communality of a test J, select, any 
which contains the diagonal entry 
and which is of order greater thar 
rank this minor must vanish. 


Y. In order to com- 
minor in the correlation matrix 
for test j but not other diagonal entries 
1 the assumed rank. By the definition of 
Its expansion isal 
known, by which the communality may be c. 

In Table 6 are reproduced the intercorrelati 


inear equation in one un- 
omputed. 


ons of eight hypothetical vari- 
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ables. The rank of the matrix is 2. Table 7 shows a minor of order 3 with one 
unknown entry, namely, the communality for variable number 1. In order 
that the expansion of the minor may vanish, the unknown diagonal entry 
must be .64. If the rank is assumed too high, say 3, it will be found that the 
coefficients of A7, as well as the numerical terms, all vanish. This indetermi- 
nacy ean be removed by assuming à lower rank. The fact that the minor of 

ate, of course, thàt the correlation matrix 


m E : 
Table ? vanishes does not demonstr 

is of rank 2. Instead of evaluating all the minors of order 3, one might pro- 
ceed to factor the matrix after all the communalities have been determined 


by this method, and it would then be found in the factoring routine whether 


Table 7 


1 2 3 


hi .96 .16 
.24 .49 .48 
.72 .67 .24 


one 


the assumed rank was correct. It is evident that this simple method is not 


stable when used on fallible data. 


2. Grouping of similar tests 
If a test battery is large and if i 


similar tests, then the tests in each constella \ 
tors in the common-factor space with relatively small angular separations. 


The communality of a test iS the square of the length of its vector. If the 
angular separations between several test vectors are relatively small, then 
a test vector on the centroid vector of the constellation 
as the lengt h of the vector. The square of the pro- 
an estimate of the communality of the test. The 
estimate will usually be slightly too low. The projection of each test vector 
on the best-fitting single vector for the constellation can be regarded as an 
approximation to the loading of the test with the single common factor 
which best describes the intercorrelations of the tests in the constellation. 

Let the communality to be estimated be denoted hè and let the tests which 
correlate highest with number 1 be denoted 2, 3, . ig 8, 8 that there are s 
tests in the group. The intercorrelations of the tests in 7 a group may ba 
assembled in the accompanying tabular arrangement, Which 15 partieular- 


t contains constellations or clusters of 
tellation will be represented by vec- 


the projection of 
Will be nearly the same 
Jection may be used as 


“з (r23) 


(ras) (ба) 


molt 
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ized for s—4. It is desired to estimate the communality hj from the projec- 
tion of test 1 on the centroid axis for this subgroup of test vectors. A slight 
underestimation may be expected, because the length of the first test vector 
will be slightly longer than its projection on the centroid 
unless they should happen to coincide, To сотрепѕ 
tion we may underestimate the communalities of t 
by writing in the three adjacent dia 
sum of the adjacent columns 2, 3, 


axis for the group, 
ate for this underestima- 
he adjacent tests 2, 3, 4, 
gonals the average of each column. The 
+++, $ may then be written 


s(Zr — En) 
Xp t 


where Zr denotes the sum of 
notes the sum of all know 
The sum of all the coe 


all known coefficients in the table and Sr, de- 
n coefficients in the first column. 
fficients in the table can then be estimated as 


s(Zr — Угу) 


En + hi) + T7. = 


and the complete sum for the first column is 


End. 


The projection of the 
first-column sum to 
table. This projectio 
as an estimate for hi 


first test vector on the centroid axis is the ratio of the 
the square root of the sum of all coefficients in the 


n should be an estimate of hi. Hence we should have, 
] 


R (Eri + hj)? 
Zr +h? + ا‎ 
which simplifies to 
(12) hy (8 DGny 


` sEr — (2s = DW x 


, Equation (12) may be written in Several equivalent forms. In computa- 
tion it may be simpler to determine only two sums, namely, the sum of the 


efi t lumn, =r1, and the sum of the known adja- 
cent coefficients in the half-table which are not in the first column or row. 


These are marked in parentheses in the present tabular arrangement. Let 
their sum be denoted Zro. Then the sum =r must be equal to 2(2ri +219). 
Substituting this expression for Xr in (12), we have the equivalent formula 


(12a) hz (8 1)(®уу? 
T nd sme * 
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where Xr, is the sum of the known coefficients in the first column and Zr, is 
the sum of the adjacent coefficients in the half-table which are marked in 
parentheses in the tabular arrangement. These formulae will be referred 
to as “centroid formulae.” 

For convenience we may list the equivalent formula for small subgroups 


of related tests. When the group contains 3 tests, we have for s=3, 


2(Zn)? 


(12b) k= 3 B (3-test formula) . 


When s=4, we have 


(12c) hè == SO (4-test formula) . 


It is interesting to note that for a doublet. when s—2 this formula reduces 
to h?=h2=ry2. A doublet is factorially indeterminate, but it will be seen 
that, when the coefficients for a doublet in a large battery do not entirely 
vanish, a graphical method may be applicable. 
3. Grouping of three tests 
A special case of the grouping of similar tests is that of using only three 
tests that are highly correlated in a constellation or cluster. One procedure 
jis to select the two tests, k andl, 


for estimating the communality of a test 0 
;. If a test battery is so construct- 


which have the highest correlations with j 
ed that each postulated ability is represented by several tests, it can be ex- 


pected that the three tests, ў, В, and 1, will be represented by test vectors 
With relatively small angular separations. The communality of test j can be 


estimated by one of Spearman's formulae, namely, 


(13) l'en 
It is here written as an approximation 


formula because the communality of test jin a test battery is not necessarily 
determined by only three tests. This would be a simple method of estimat- 


a ларе ; от except for the fact that it is un- 
ing the с ties in practical W ork excep Я 
е communali It can be used as опе estimate of the 


stable because of sampling errors " 3 
communality if it is combined with other more stable estimates. It will be 
referred to as a “3-test formula.” 


4. Highest coefficient in each col 
(13) for 
tion in the estimat 
orrelations in 


Which is equation (15) of chapter xii. 


umn 

estimating the communality of a test sug- 
e. The numerator contains the 
the column for test j. The de- 


Inspection of equation 
gests a further simplifica 
product of the two highest © 
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nominator is the intercorrelation of the two tests so selected, namely, 
k and l. If these coefficients are of the same order of magnitude, then the 
estimated communality of test J will be nearly equal to the highest inter- 
correlation in column j. This is the method that has been used in a large 
number of studies in which the factoring was done by one of the centroid 
methods, adjusting the diagonal values of the residuals after each factor 
had been computed. This simple method of estimating communalities is 
useful only for large correlation matrices. It is not applicable to small tables. 

The principles of No. 3 and No. 4 can be combined in another computing 
formula. Let test 1 be the test whose communality is to be estimated, and 
let tests 2, 3, and 4 be several other tests which have high correlations with 
test 1. The coefficients may be shown in the accompanying tabular arrange- 


M re ms ra 
№ rea ra 
Tis Ta kè ru 
Tis Tog ra ht 


[M 


х 2.64 2.67 2.90 2.88 11.10 
ment for four 
coefficient in 


tests. If, as a first estimate for each test, we record the highest 
he column for this subgroup of four tests, we have the numeri- 
cal entries which were obtained from Table 6. The tests 2, 5, and 6 are 
those which correlate highest with test 1. The first-column sum is, then, 
(Eri + 0), where Er, denotes the sum of the known coefficients in the first 
column and tı is the highest coefficient in that column as recorded in the 
diagonal cell. The sum of all the coefficients in the table is (Sr-++Sd), where 
Zr is the sum of all known coefficients. By the centroid formula we then 


have 

(14) hye "+ 
so that 

(15) hg St + д) 


Zr + X 
For test 1 we then have the numeric 
:58 аз an estimated communality. 
tion (14) is a centroid formula. 


al estimate (2.64)?/11.10, which gives 
The true value in this case is .64. Equa- 


5. Linear dependence of rows and columns 

If the rank of a correlation matrix is 7, 
linearly in terms of any r independent row. 
this principle into a method of computi 


then any row may be expressed 
s. It may be possible to generalize 
ng the communalities for fallible 
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data. In using this method one assumes a rank for the correlation matrix, 
and it is also essential that the rows chosen for the computation be linearly 
independent. 

The idea of using linear dependence of rows and columns of the correla- 
tion matrix could perhaps be used for estimating the communalities if the 
matrix were divided into four or into nine sections. The method can be out- 
lined with the adjoining table showing a correlation matrix divided into nine 


Ru Ре ds 
Ra Re Ras 
Ra Rs» Rs 


square sections. Let us assume that the tests have been divided into clus- 
ters by inspection of the correlations. Each cluster should be represented 
by at least one test in each of the three groups into which the tests have 
been divided. The notation in the table means that Ress, for example, is 
the square section showing the correlations of the tests in the second group 
with the tests in the third section. With this prearrangement of the tests 
in three groups and with the assumption that the rank does not exceed 
one-third of the number of tests, it should follow that the third group of 
columns can be represented by a linear transformation of the second group 
of columns, and similarly for the rows. 1f T denotes a linear transformation, 
we should write RaT = 21, КТ = Roo, R3sT = R23. Since the sections Ps; 
and Rəı are known, the transformation T' could be determined and hence 
the other sections. The side correlations of the symmetric sections so com- 
puted should not deviate far from the experimentally given values in the 
three diagonal sections, and the communalities would then be estimated. 
This idea has not been developed into a practical computing method for the 
ight be done. * 


communalities, but it m 


6. Sectioning of the matrix 

The unknown communalities can be solved for by sectioning the correla- 
tion matrix. This method has not been used because it requires more com- 
putation than the simpler methods that have been found adequate in prac- 
tice, and it is not à stable method for fallible data. The sectioning theorem 
will be given here because of its interest in relation to the correlation matrix. 
If any matrix of rank r is sectioned into a composite square matrix of order s, 
where 87 r, then the determinant of the composite matrix vanishes. | 

The matrix will be said to be sectioned when the columns have been di- 


vided into s groups, and when the rows have also been divided into the same 


itten the idea has been developed into a computing for- 


* Gi is chapter was WT д 
Since this char tion of this chapter on “Further Studies of Linear De- 


mula, as described in а later sec 
pendence.” 
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number of groups. Let r=2 as an example. Since s>r, we may let s=3. 
Then the n columns of R will be divided into three groups of p, (q— p), and 
(n—4) columns, respectively; while the n rows of E will be divided into 
three groups of t, (u—t), and (n— и) rows, respectively. The matrix R will 
then be sectioned. 

'The composite matrix will be defined as the square matrix of order s in 
which the entries are the sums of the elements in the corresponding parts of 
the sectioned matrix. The example of Table 8 illustrates the formation of a 


Table 8 


composite matrix. This 4X5 matrix is of rank 2. It has been sectioned into 
a 3X3 square matrix by arbitrarily dividing the columns into three groups 
of 2, 2, and 1 columns, respectively, and by arbitrarily dividing the rows 


into three groups of 1, 1, and 2 rows, respectively. The composite matrix is 
shown in Table 9. Its determinant vanishes. 


Table 9 
20 14 2 
110 30 17 
78 42 6 


The proof of the theorem will be written fe 


generalized for any rank. If Ris of rank 2, it is possible to find two rows that 


are linearly independent. Let these be the first and second rows. Then the 
elements of the jth row can be expressed as a linear function of the first two 
rows so that 


or rank 2, but it can readily be 


Ta = тту + mra, 


= Mle + ә», 


(16) 


| Tj = тту + MT. 
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It is evident that the sum of the first p entries of row j can also be expressed 
as the same linear function of the corresponding sums in the first two rows. 
We then have 


(17) 


Similar summations may be written for the other two groups of columns so 
that 


(18) 
EN RN ч. 
У Tj = M У тк + Mes > Toka 


— 
0 k=(at+)) k=(q+1) 


1 


These summations may be represented in an п ХЗ matrix as shown in 
Table 10. Since each of the rows can be expressed as a linear function of the 


Table 10 
p 
E 
> гь = bu 
— 
Т 
p 
RN 
b тәк = ba 
ہے‎ 
1 
р 
C 
Nn = 
— 
L 
» d. = 
k р 
N Tak = Ва У Tak = bnz S Tue = bus 
P і — 
1 p+1 qr 


it follows that the rank of this nX3 matrix is also 2. The col- 
d that the third column of this matrix may be ex- 
first two columns. This reduction by columns is 
similar to the reduction by rows that has been described. This reduction 
by columns gives а 3X3 composite matrix whose rank is 2, and hence its de- 
terminant vanishes. If the rank of R is equal to or greater than the order s 


first two rows, 
umns may be so arrange 
pressed in terms of the 
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of the square composite matrix, then the determinant of the composite does 
not necessarily vanish. 

The communalities can be solved for by sectioning, if this is done in such 
a manner that one row and the corresponding column are taken as sections. 
Their intersection will be a section with a single entry, namely, the commu- 
nality to be determined. 


7. Expansion of principal minors of order (n—1) 

It is possible to write n principal minors of order (n—1) in a square 
matrix of order n. If the expansion of each of these n principal minors of 
order (n—1) is set equal to zero, the rank of the matrix is assumed to be not 
greater than (n—2). This follows from the property of a Gramian matrix 
that if all of its principal minors of order m vanish, then the rank of the 
matrix does not exceed (m—1). Since there are n principal minors of order 
(n—1), their expansions give as many equations as there are unknown diag- 
onal entries. A unique solution is obtained if the inequality (10) is satisfied. 
If this inequality is not satisfied, there should be no unique solution. In this 
method it is not necessary to know the rank. These considerations are of 


some analytical interest, but they do not seem to lend themselves to com- 
puting purposes, 


8. Expansion of principal minors of order (r--a) 

This should be a Special c. 
It is not necessa 
taken larger th: 
method require 
minors be such 


ase of the preceding method but less laborious. 
ry that the rank be known, but it is assumed that (r-+a) is 
an the rank. The simplest case is that in which a=1. This 
8 that the number of tests covered by the expanded principal 
t às to satisfy inequality (10), even though all the tests in the 
correlational matrix are not utilized, The development of this type of anal- 
ysis would be of interest, but so far it has not led to practical computing 
methods. 

9. A summation formula 


А es the test whose communality is to be determined be denoted number 
, and let number 2 be the test which correlates highest with test 1. These 


two tests will usually have a small angular separation, and hence their in- 
tercorrelations should be nearly proportional Hence 


= 
afi — Tis 2 

(19) = PET 
Ble Fig” Wt 


where Er; and Xr, are the sums of the known coe 


Be ; ficients in the two columns. 
This gives the approximate computing 


formula 


(20) hg = Tr — т) 


Gras — n) ' 
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which may be referred to as a “summation formula.” Its source of insta- 
bility is the single coefficient гу», whose variable error influences directly the 
estimate of hj. This formula makes the assumption that the two columns 1 
and 2 are of unit rank and hence that they are proportional. 


10. Spearman’s formula 
The summation formula of equation (20) can be generalized for a group 
; including test 1, which may be assumed to be nearly 


of four correlated tes 


collinear. From (20) we may write for each of the three tests 2, 3, 4, 
(21) m ru(Xr — nus) _ ra(rı — na) _ ru(Xn — ru) 
m п Sra — ras) Dr; — ris) (Sry — rı4) 


where Yr, is the sum of the known coefficients in column x. From these 
three expressions we write 


ris Eri — nis) c ri(Eri та) + ru(Eri — ти) 
(Irs — ris) + (Srs — та) + (Er — гы) 


(22) h 


which simplifies to one of Spearman’s formulae for the single common fac- 
tor, namely, 
rı) 


ar = 


== (X 


— 95р cr 


(23) 
which is the same as equation (51) in chapter vii in different notation. 
Here Xr is the sum of all known coefficients for the subgroup and Zr? is the 
sum of the squares of the coefficients in the first column. This formula of 
Spearman should give a good estimate of the communalit y, although it was 
not originally written in the present context. The formula assumes ap- 


proximation to unit rank for the subgroups. 


11. Grouping of four tests 
Spearman's 3-test formula, equation (13), 
four correlated tests where test 1 is the one whose communality is to be 


estimated. In the 3-test form we have 


can be adapted to the case of 


in the relations 


and this can be extended to four te 


Trelis e 713714 e 7127714 


T34 
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These three formulae can be combined into a single 4-test formula by tak- 
ing the geometric mean of the three expressions, and we then have 


(26) 


12. Graphical methods of estimation 


If two test vectors (1 and 2) are nearly collinear, the correlations of their 


respective columns should be proportional to the ratio of hi/hs. We should 
then have 


s 72; _ Reh; cos фә; 
(27) al بای‎ аА: ЙЧ 


Тї; Ау cos фуу' 


and, since the two cosines should be nearly the same, we have 


(28) ту _ а 
ту № 

or 

(29) he 


Тау =т= Pip = 89171. 
J hi 1 


Hence there should be 
Since the corre] 
proximately, 


a linear plot of rs; against rı; through the origin. 
ation ту should be nearly equal to Ай», we should have, ap- 


(30) 


and hi = risa, 


by which the communalities can be estimated. When the plot is clearly 
linear through the origin, this method of estimation is good except for the 
variable error in гз. A clear linear plot indicates that the two test vectors 
are nearly collinear. This method of estin 
one in dealing with a doublet, in w 


coefficient in the column. 


If two tests, 2 and 3, are coplanar with test 1, a computing formula ean 
be written for estimating the communalities by plotting rs;, їз; against гу. 
The plot should then be linear but not ordinarily through the origin. From 
the slope and one of the intercepts one could then determine the linear com- 
bination of columns 2 and 3, which reproduces column 1, This coplanar 
method is of some interest, but it is not, in this form, readily applicable. 
The collinear form of graphic estimation is often quite useful, 


nation is probably the best: 
hich case there is only one appreciable 
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Summary of methods 

When a column of the correlation matrix contains a number of coeffi- 
cients that are quite high, the estimate can be made with one of the formulae 
which include the correlations of several tests. In general, we should expect 
instability with fallible data when the error in the estimate is directly af- 
fected by the error in a single coefficient as in equations (13), (20), and 
(30). 

When a column contains only one high coefficient, none of the computing 
formulae apply because we then have essentially a doublet. The estimate 
might then be made by the graphical method. A pure doublet in which 
there is only one non-vanishing coefficient in the column, while all the rest 
are zeros, is factorially indeterminate. Theoretically, this case is the same 
as that of a correlation matrix for only two variables. There is only one 
correlation which is insufficient to determine two communalities. In prac- 
tical work one seldom finds a pure doublet, but one does find approximations 
to them. If a doublet is to be retained in a battery, one can sometimes make 
a good estimate graphically if the remaining coefficients do not entirely 
vanish, Another procedure is to set both communalities equal, in which case 
they are both equal to their correlation, the highest in the column. 

If the factoring is to be done by a method with adjustment in the diag- 
onals after each factor and if the battery is large, say, more than twenty or 
thirty variables, then one may record the highest coefficient of the residual 
column in each diagonal cell. This procedure is not adequate for small cor- 
relation matrices. For very small tables of, say, less than ten variables, one 
may need to repeat the factoring several times to obtain good estimates of 
the communalities and an acceptable factor matrix. 


Uniqueness of communalities 

We have considered the problem of ascertaining whether there are enough 
independent correlation coefficients in Æ to determine the factor matrix for 
r factors. This problem is solved by the relations of the inequality (10), 
which gives the maximum number of common factors that can be deter- 
mined from the intercorrelations of n tests. In the theorem for the inequal- 
ity we stated that for any given number of common factors (r) to be deter- 
mined by a given number of tests (n) it is necessary that the inequality be 
satisfied. The theorem does not state that the satisfying of the inequality 
(10) is sufficient to insure that r factors can be determined from the inter- 
correlations of n tests. In deriving the inequality we merely counted the 
number of independent parameters in the correlation matrix and the num- 
ber of independent parameters in the desired factor matrix. The relation 
between these two counts does not insure that the diagonal values will be 
positive numbers between 0 and +1, and it does not take into consideration 
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the possibility of multiple solutions with real or imaginary diagonal entries 
in the correlation matrix 


If a correlation matrix is factored by the diagonal method with unit com- 
munalities (chapter ii), a factor matrix сап be obtained which accounts for 
the intercorrelations exactly with not more than n factors. If n factors are 
required to account for the intercorrelations, then the last column of the 
factor matrix contains only one entry. Hence it may be regarded as a spe- 
cific factor for the last test. We therefore have the theorem that there ex- 


ists at least one set of communalities between 0 and +1 for which the rank of 


the reduced correlation matrix is less than the number of tests. Every correlation 
matrix can be accounted 


for by not more than (n — 1) common factors. 
In making an exact solution for the communalities, one would take a 
trial rank 7 and then expand minors of order (r--1) which should vanish. 


| | 

4 
ا A‏ ا 
x‏ — ي م 


Ficung 1 
These minors should be so selected th 
diagonal cells. In Figure 1 wi 
the principal diagonal. Let 


at they contain a minimum number of 
€ have represented the correlation matrix and 
! d ABCD be a minor which is so placed that it 
contains û minimum number of diagonal cells. Let the number of columns 
which do not contain any part of the minor be denoted b. The number of 
columns b then is ў 


E A 


and hence the number of diagonal cells outside the indicated minor is 


2(%—т— 1); 


so that the minimum number of е 


f ) ommunalities M that c. 
minor is 


an be placed in the 


(31) М =n — 25 کن‎ 1) 
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or 


(32) M=2r—-n+2. 


When J <0, a number of minors of order (r4- 1) can be taken which contain 
a particular communality and no others. If M € 1, then a minor can be so 
placed that it will contain only one communality. By expanding such a 
minor and setting it equal to zero, we have a linear equation with one un- 
known, namely, the desired communality, which can then be numerically 


determined, We then have 


(33) бе = for DEST 


The number of factors is then less than half of the number of tests. When this 
condition obtains, there is a single analytic solution from the selected 
minor. If the communality so determined should exceed unity, then the 
rank of the correlation matrix must be assumed to be higher than 7 and a 
higher trial value for the rank must be taken. When r is taken so that 
M «0, the consistency of the several possible determinations of the com- 
munalities must be examined. 

Multiple solutions for the communalities are of special interest, and they 
may arise when r is taken so large that the minor ABCD in Figure 1 con- 
tains two or more communalities. If we set M =2, we have r=n/2, so that 
the minor ABCD of order (7+1) contains two communalities. When this 
minor is expanded and equated to zero, we have a quadratic which may in 
some circumstances give two real solutions for the diagonal value between 


0 and 4-1. 


This problem has been investigated by E. B. Wilson and Jane Worcester* 


in several papers. They exhibit as an example a correlation matrix with 
Their example is reproduced in Table 11, which shows the 
rix and two factorial resolutions, both for three factors and 
ommunalities. In Table 12 we have another example 
from the same authors which reduces to only one set of communalities. A 
second solution gives diagonal TEN that do not lie between 0 and +1. A 
third example from the paper, The Resolution of Six Tests into Three 
General Factors,” is shown in Table 13, for which the authors find no set of 
communalities between 0 and +1 that reduce the rank to 3. A higher rank 
must then be assumed for this example. It is pointed out by the authors 
= Vorcester, “The Resolution of Six Tests into Three Genera 
sc ш Iy National UN K estas: TE No. 2 Sei 
1939), 73-77; and “Team Tests for Mp d d a ie “ pote 7, July, 1939, pp. 
ew of Crossroads in the Mind of Мап [by T. L, 


358-64: : D Vilson, “Revi s à 
re ae Goneral Psychology, II (1929), 153-74, esp. 156. 
Ё У}, Јонг T 


n=6 and r=3. 
correlation mat 
with different sets of ¢ 


Correlation Matrix 


Table 11 


Factor Matrix 


4 


5 6 I II ш h 
1 hî mU. 18 48: э g 8 0 0 ‚64 
2 56  .hi 20 .66  .51 .86 C 6 0 ‚85 
Б] .16 20 hå AS: OF 28 .2 s =q .06 
4 48 .66 „1а hî 0 72 .6 .4 چس‎ .56 
5 .24 .51 И .30 hi 41 Б] Б] A ‚50 
C Q4 0.86 .293 э д м Be MEE 
Alternative Factor Matriz 
y I п D 
‚6524 0 .425616 
. 8584 - 4068 . 902308 
+2453 — .0259 . 063569 
-7358 -0699 . 546923 
-3679 14774 - 386667 
- 9810 ‚0441 . 998000 
Table 1.2 
Correlation Matrix Factor Matrix 
E а. Kin Nee cm So uro ли y 
L| М бё 18. из ч йй Pastor ambi pie tent 
2| 56 a Ug Ue Ej m 2 у 0 P Factor matrix unique 
2I 8 320: ла log 27 129 ep Vu 20] 
“А 28 08 JOR "gr ^an “ao 6 .4 сы д [56 
б a p^ -27 30 h т d B 4 .50 
ЕБИНЕ ЕИ VATI OE IR. 55) cho. 108 
T'able 13 
Correlation M. atriz 
1 A eat 1б м& о 64 No solution in 
2 .56 18. .20 66 51 .86 three factors 
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(“Team Tests for Generals and Specifics,” p. 362) that, when the number of 
tests is much larger than the number of factors, then M is negative, and the 
question of multiple solutions does not arise. 

It is of interest to note the number of independent conditions in the cor- 
relation coefficients for the resolution of n tests into r common factors. This 
is the difference between the number of independent correlation coefficients 
and the number of independent parameters in the factor matrix. It can be 
obtained from the inequality (7). Denoting the number of independent 
conditions in the correlation coefficients T, we have 


(n—1 (7—1 
(34) pa Н capit 
which has been written by Wilson and Worcester (in “The Resolution of 
Six Tests . . . . and “Team Tests for Generals and Specifics") in the form 
(35) T = 3(n—r)? — $n). 


When the number of independent conditions in the correlation coefficients 
is zero, the resolution is unique in the sense that no arbitrary parameter is 
involved in the solution but not necessarily in the sense that there is only 
one solution. In the numerical examples of Tables 11, 12, and 13, T —0, 
since n= 6 and r 23; but M —2, and hence multiple solutions may arise. In 
one example the solution is unacceptable because of diagonal values that 


exceed unity. i | | | 
These relations are of importance in dealing with factor problems in 


which the rank is as high as half of the number of tests. If the test battery 
can be so constructed that the rank is not more than one-fourth or one- 
third of the number of tests, then the question of multiple solutions for the 


communalities does not апе  — 2 ] 
The uniqueness of communalities is discussed further in a later section 


of this chapter. 


Adjustment of communalities for each factor 
In factoring à correlation matrix, one may adjust the communalities 
after cach factor if residuals are computed. The simplest method of making 
this adjustment is to record in the diagonal entry the highest correlation or 
residual of the column. If the highest coefficient of the column is negative, 
one records this value in the diagonal with positive sign. This procedure is 
elation matrices, and it can be used as a first approxima- 


useful for large corr : 1 
йол ЫЕ ‘ere of a small table. The procedure will be illustrated by a 


numerical example. 
In Table 14 we have 
Although method No. 4 woul 


a small correlation matrix of order 5X 5 and rank 2. 
d be used only as a first approximation in fac- 
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toring a correlation matrix as small as this, it will be used here asa numerical 
example of the effects. The true communalities are shown in the diagonal 
cells of R. The first factor matrix was obtained with the group centroid 
method, using the known communalities. The second-factor residuals were 
then identically zero, and the obtained communalities were the true ones, 
as shown in the table. 

The second factor matrix was obtained by using the highest coefficient in 
each column as the estimated communality. The discrepancies for a small 
correlation matrix are then often marked. In the fifth column, for example, 
the highest coefficient in the column is .72, whereas the true communality 


Table 14 


Correlation Matrix of Rank 2 


1 2 3 4 5 


(64) .56 I .24 .72 

| :56 (65) .88  .49 67 

| -16 .38 (.40) .48  .24 
| 


24 .49 48 (58) .34 
Л ЧТ 294 034^ (189) 


Factor Matriz Co mputed with 


Factor Matriz Computed with 
Known Communalities 


Unknown Communalities 


I II te | I п l^ Diserepaney 
1 | .680 —40 быр M — 447 -054 
2 | .800 —.029 650 2 —.029 -010 
3 -486 .404  .399 3 .442 -056 
4 | .624 437 1560 1 .400 —.062 
5 | .817 —.390 80 5 —.875 —.058 


is .82. Using these estimates, 


we obtained column I of the second factor 
matrix by the group ce 


ntroid method. The correlational residuals were com- 
puted, and the diagonal entries were adjusted by recording the highest resid- 
ual in each column. The second faetor column II was then obtained, In 
the last column are shown the resulting discrepancies between the obtained 
communalities and the true values, Tt will be seen that the discrepancies 
are smaller, the largest discrepancy being .06. For large correlation ma- 
trices the discrepancies between obtained and true communalities are stil] 
smaller, 

In Figure 2 we have the true configuration of the fiv 


e test vectors ob- 
tained from the first factor matrix of Table 14. These test vectors are repre- 


sented by solid lines in the figure. In the same figure we have superimposed 
the configuration obtained with the estimated communalities. These vec- 
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tors are shown in dotted lines. It will be seen that the configurations are es- 
sentially the same in spite of the coarse method of estimation of the diagonal 
entries for each factor. The second factor residuals do not vanish identically 
in this case, but they are so small as to discourage the attempt to write a 


third factor. 


FIGURE 2 


actorial analysis it is desirable to cover the domain to be 
investigated as completely sible with a large number of related vari- 
ables, which should be experimentally independent but correlated and rep- 
ative of different. aspects of the domain. When a study has been 
planned in this way, it will nearly always happen that every test vector lies 
near some other vector in the system. This arrangement is desirable for 
several reasons, principally in order to make it possible to discover a simple 


In planning a f 


as pos 


resent 
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structure if it exists in the domain. The same plan will also make it almost 
certain that the computing formulae, including the collinear graphic meth- 
od, will be applicable in the estimation of communalities at the start so that 
the investigator is free to use any factoring method, including those which 
avoid the computation of residuals. The most troublesome correlation ma- 
trices are those which were not planned for factor analysis. In such cases the 


rank is often a considerable fraction of the order, and the computational 
problems then become more difficult, 


Further study of linear dependence 


Since this chapter was written, the method of estimating communalities 
by linear dependence of rows and columns (method No. 5) has been investi- 
gated further, and two papers have been written on this problem by А, А. 
Albert, of the Department of Mathematics at the University of Chicago. * 
The method of estimation of communalities by linear dependence of rows 
of R will be described here with computing formulae and a numerical ex- 


ample. 
Rt S Re Ris 
Ry Razr Dy Аз 
э, Re % * D, 
EMEN з 


FIGURE 3 


In Figure 3 we have represented 
described earlier in this chapter, 
Ras, are here defined with zero di 
onal elements are given in the di 


the correlation matrix, partitioned as 
The symmetric submatrices, Ri, Ies», and 
agonal elements. The corresponding diag- 
agonal matrices Dı, Ds, and Ds. It will be 
assumed that the correlation matrix has been inspected for clusters and that 
the number of distinguishable clusters represents a good estimate of the 
rank. In assigning tests to the three groups into which the correlation ma- 
trix is sectioned, it will be assumed that one test from each cluster has been 


assigned to the first group and that one test from each cluster has been as- 


* “The Matrices of Factor Analysis," Proceedings of the National Academy of Sciences, 
XXX, No. 4 (April, 1944), 90-95; and “The Minimum Rank of a Correlation Matrix,” 
ibid., No. 6, June, 1944, рр. 144-48. 
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signed to the second group. The result will be that the submatrices, Ёз, 
Ro, Ri, and Rss, are square and of order r where r is the number of clusters. 
The matrix Ёз» is square and of order (n—2r). The remaining submatrices 
may or may not be square. The underlying assumption is that if all the 
factors are represented in each of the first two groups, then the correlations 
of the third group can be accounted for as linear combinations of rows of 
cither the first or the second group. This is the central idea of this method of 
estimating communalities. If a doublet cluster is found consisting of two 
tests which show unusually large correlation, then one test from the doublet 
is assigned to group 1 and the other test of the doublet is assigned to group 2. 
A doublet is fundamentally indeterminate as to factorial analysis, and it ap- 
pears as an ambiguity in the rotational problem. 

When the correlation matrix has been arranged in this manner and as 
shown in Figure 8, we can express the rows of Rs: as linear combinations of 
the rows of Ra. The transformation can be written 


(36) Alis = Ra, 


from which we can determine the transformation matrix A, since all the 
elements of the two submatrices are known and contain no communalities. 


We then have 


(37) A = Ва Е). 


It will be found that Rs, is the only inverse that needs to be computed. If it 
is found that Ёз does not have an inverse, then either the assumed rank is 


too high, so that its determinant vanishes, or the clusters have been chosen 


inappropriately. . Р ў 
When the transformation matrix A has been determined, we express the 


rows of Ras as linear combinations of the rows of Rss by the same transforma- 


tion, and we then have 

(38) Аз = R33 + Рз. 
The side correlations of this matrix must agree with those of the subma- 

tis Ris. If this is not the case, then the rank has probably been assumed 

\ Tf this agreement is obtained, then the diagonals are those of Ras. 

s of Rss are then known. 

ansformation to Rs», we have 


22, 


too low. 1 
The communalitie 
Applying the same tr 
(39) A(R: + De) = Ras, 
or 


(40) Rz — AR» = Ар». 
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We compute the left member of this equation. The ratio of each column 
sum to the corresponding column sum of А is then the desired commun 
for the corresponding column of Rəs. * 

The same process is carried out for the remaining set of three submatrices. 
The rows of Rss can be expressed as linear combinations of the rows of Ryo 
and we then have 


ality 


, 


(41) ВЕ = Ras, 
so that 
(42) B = Ry Rī. 


We can write this equation in terms of the 


inverse already computed, and 
we then have 


(43) B' = RẸ Rag 


so that 


(44) В = (Rj! Ray). 


Applying this transformation to Ris, we have 


(45) B(Ru + Di) = Ry 
or 

(46) ВЕ + BD, = Ry Л 
so that 

(47) BD, = (Ry — BRy) . 


The ratio of the columns of the right member to the columns of B give the 
corresponding estimates of the communalities of Ёз. 


Table 15 shows the computations for this method of estimating com- 
munalities which is illustrated on a correlation matrix of order 6 and rank 2. 
The summarized computations follow the outline as shown. 

A limitation of this method for experimental data is that the rank is 
never exactly r where r <n and that the determinations will therefore be ap- 


* In practice one can save com 


putational work by finding only the column sums be- 
cause these enter into the determi 


nation of elements of Ds. 
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proximate. Even when the approximate rank r is adequate, as far as corre- 
lational residuals are concerned, the present method may, nevertheless, 
show discrepancies in the estimated communalities for experimental data. 
The method has not yet been tried extensively enough on experimental data 
to show whether it is preferable to the simpler methods of estimation. The 
method is certainly amply justified for correlation matrices in which the low 


rank is exact or very closely approximated. 
In the two papers on the communality problem Albert defines the ideal 
rank of R as the largest order of a non-vanishing minor of R which does 


Table 15 
1 2 3 3 6 
Tr à 72 61 92 —.26 
2 00.80 68 20 
" 3 “80.00 54 .40 
Rens 4 OS 54 00 —.39 
5 :20 — .4Q 39 .00 
6 68.69 .55 .21 
Ry = Р RAR» = — 1.8181 — „0909 
1 92 (43) 1 8863 9318 =B’ 
ryt = 2.1170 2.4906 — — з 
Ts 2 8612  —1.8091 H-—1 8181 1.8863 
RRO es — .0909 “9318 
s,——1.0000 2 8181. 
7 1933 1.0274 
= 1.1233 — 1.02 
= 1. 6086 1781 
ج ی‎ ————À 
„= 1.8219 = .8403 Ка = ВЕ = —1.6181 1.5090 = BD, 
Lad Е (47) — 080% 7454 
namm , s, — 1.6090 2.2544 
JU sus 850 .210 2 Aid Ds 
ARs В at 580 
npe he hy 
sa/sı= 8900 8000 
ДИШЕ 7 
— .8238 
he UH hi hi hi hè м hy 
sa/sı= -8500 ‚9700 .890  .800  .850 970 850 580 
$2, p == 
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not contain diagonal elements. He proves the interesting theorem that if 
the ideal rank r of R is also the rank of the submatrices R3; and Ts, then there 
exist unique communalities such thal the resulting correlation matrix В. has 
rank т. This is an encouraging finding because it has been assumed in the de- 
velopment of multiple-factor analysis that there exists a unique set of com- 
munalities which are consistent with the experimentally given off-diagonal 
correlations. The existence of such a set of communalities merely maintains 
the rank of the off-diagonal entries, which Albert has called the ideal rank 
of R. It had previously been pointed out by E. B. Wilson that if the number 
of tests is much larger than the number of factors (n7 г), then the question 
of multiple solutions for the communalities does not arise. 


Experimental comparison of communality estimates 

Since this chapter was set in type Mr. Frank Medland has completed a 
Master's dissertation, entitled “An Empirical Comparison of Methods of 
Communality Estimation.” Nine methods of estimating communalities 
were tried on a 63-variable correlation matrix from Psychometric Mono- 
graph No. 2. These communality estimates were compared with com- 
munalities that were computed by the centroid method, with diagonal 
adjustment after each factor. Medland found that the centroid formula 
of equation (15) gave the most consistent communality estimates. Prac- 
tically the same results were obtained by the graphical method that is 
represented by equation (30). Since the centroid formula of equation (15) 
1s computationally the more convenient of these two methods, it may be 
considered the preferred method. 


CHAPTER XIV 
THE SIMPLE-STRUCTURE CONCEPT 


The score function 

Simple structure has been briefly mentioned in previous chapters, but it 
will be considered here in more detail as to its significance and implications. 
The principles of simple structure are fundamental in making factor analy- 
sis a scientific method rather than merely a method of statistical condensa- 
tion, and it is therefore one of the central themes in factorial theory. Some 
writers on factorial theory have not yet accepted this concept as a powerful 
aid in exploratory factorial investigation. It is surely not the only way to 
resolve the problem of finding a significant reference frame, but it has so 
far yielded some very illuminating results. 

Consider a set of measurements S;; of individuals û in traits j. The meas- 
urements selected for factorial study are correlated, and hence they can be 
assumed to have some parameters in common. Let the measurements be 


expressed in terms of these parameters by the equations 


вы = Qi Voip -e y Tpi ee ‚Ж аў 
Soi = altii Teis eeo Tpi es ыў), 

1 
a) Sji = dis Veir eo Ypi ee sla 
Sai = Onlin Tos ea Yon a Tri) - 


Here we have each measurement sj; for individual i expressed in terms of r 
which are descriptive of individual 7. The functions фу are 
in general non-linear, and they are unknown. They may be assumed to be 
unique for each measurement j. If a measurement S; does not depend ona 
parameter X, then that parameter is absent in the function ¢;. Every pair 
of measures, j and Ё, will be assumed to have one or more parameters in 
common if rj, z20. 1 vj —0, they may or may not have parameters in com- 


mon. 
For any given list 0 


parameters tpi; 


f n measurements Sj, let there be q parameters in- 

volved. Some of these parameters will be unique in that they are not in- 

volved in two or more of the score functions Sj. We limit the analysis to the r 

parameters which are involved in two or more of these. score functions and 

which determine correlation between the scores. It is this set of r parameters 

that determines the side correlations and the intercorrelations rj; where 
319 


320 MULTIPLE-FACTOR ANALYSIS 


j#k. The (q—7) unique parameters influence only the diagonal correlations 
rjj, Which are eliminated from the analysis in order to make the problem 
soluble. If we knew all the parameters and all the functions Фу we should 
probably be able to design a set of measurements for any set of parameters 
so as to make a determinate solution, irrespective of whether the parameters 
were common or unique. But when the parameters become known, there 
will be no provocation for factor analysis. It is our ignorance about the 
parameters and the score functions that prompts us to use the exploratory 
factorial methods. In our present state of knowledge we are likely to add 
unique parameters with every new measurement, so that we are likely to 
have q7 n, a situation that involves more parameters than there are meas- 
urements for each individual, 

Any particular list of 7 parameters defines a domain, which includes all 
possible measurements that are determined in whole or in part by these param- 
eters. In practice, a domain is defined by a field of related measurements, 
since the underlying parameters are unknown. A domain is described in 
practical work in terms of the content and nature of the measurements, such 
as auditory perception or visual illusions or speed performances. The com- 
plexity of a measurement Sj is the number of parameters X, that it involves 
and which it shares with one or more other m sasurements in the battery. 
Hence the complexity of a test or measurement is dependent on the test 
battery in which it is being analyzed. The comple: ty of a test j is, in part, 
dependent on the test battery. When we speak of the complexity of a single 
test apart from a test battery, we refer to the total number of parameters 
in the score function S;. 


M 1 gu n э ۰ H 1 
Consider a table of order nXr, in which each cell represents a parameter 
for a particular test, I% 


Zach row represents a function Ф, and each column 
represents a parameter. The complexity of a battery of n tests is r, and it 
should be the aim of the investigator to choose a set of n te: 
ments for the domain which he is studying so that n>r. His scientific in- 


tuitions and hypotheses are his chief guides here. If the score functions were 
known, we could mark an x in each cell (jp) to show that a parameter X, is 
involved in the score function 


} neon фу, and we could leave blank cach cell (jp) in 
which the parameter X, is absent in the score function фу. We call such a 


table a factor pattern (see chapter ix for different types of factor patterns). 
) If an analysis is to be made by the principles of simple structure, then the 
Investigator gambles that the complexity of each test or measurement is 
less than the complexity r of the battery as a whole. He hopes that the test 
complexities are considerably less than that of the battery. If the test com- 
plexities are less than т, then there will be one or more blank cells in each 
row of the factor pattern. Such a pattern will be considered as evidence of 
an underlying simple order in which each parameter or process in the domain 
1s not involved in every test measurement. If the parameters are uncorre- 


S or measure- 
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lated in the experimental population, then the pattern will be indicative of 
an orthogonal simple order for that population. If the parameters are corre- 
lated in the experimental population, then the pattern indicates an oblique 
simple order. The question of whether the parameters are correlated or un- 
correlated among the subjects who happen to be used for studying the do- 
main is, in general, a secondary matter when compared with the scientific 
problem of discovering an underlying order in the domain. 

The score functions S; may be represented by a surface, as in Figure 1. 
In this figure we have considered only two of the parameters, X; and X», as 
well as the score S;. If the score S; is independent of Xs, then the surface 
5; will be a cylinder parallel to Х,. (А cylinder is, of course, not necessarily 
circular in section. I is а surface parallel to an axis.) An individual with 


parameters 21; and аз: will have a score sj; as shown, and this score will be 


FIGURE 1 


nt of the parameter Xs. This reasoning can be extended to n di- 
mensions. If the surfaces could be constructed or otherwise represented, 
al forms of the surfaces could be taken as clues for locating the 
especially if a number of such surfaces were found which 
axes in this manner. If a reference frame could be de- 
surfaces, it could be expected that the axes, so 
determined, would represent parameters for which some physical or opera- 
tional interpretation could be made. It would not necessarily follow that 
such a reference frame would be the only feasible one, but the simple order 
indicated by these axes should be illuminating in the exploratory study of a 
domain for which no hypotheses are available. Each axis should represent 
some underlying process, parameter, or category which may or may not have 
been suspected in setting UP the test battery. Unfort unately, these surfaces 
are not immediately available in the measurements and their relations, so 


imati 5 1. 
that approximations must be used 


independe 


the cylindric 
reference frame, 
defined the same 
termined by the cylindrical 
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There are certain characteristics of the surfaces S; which may be sur- 
mised by psychological considerations. Most of the factorial studies have 
been made with those mental traits which can be called “abilities.” An 
ability may be considered to be any trait which describes how well a person 
can do something. It has been observed that all such traits have correla- 
tions that are positive or zero, and this seems to be the rule for the most 
diverse abilities. The parameters that underlie individual differences in the 
mental abilities may be tentatively assumed to be of such a nature that they 
are covariant with the overtly manifested abilities. If that should be found 
to be acceptable, then we should be able to infer that the surface S; rises 
with an increment in any of the underlying parameters Y,. By these psy- 
chological considerations we should infer that 


(2) ЭХ 


for those traits that are classified as abilities. In other words, the surfaces 
S; for mental abilities may be assumed to be monotonic increasing functions 
of the parameters X,*. If this inference is acceptable, then it should be pos- 
sible to represent all the surfaces S; reasonably well with second-degree 
equations for the mental abilities, barring what might seem now to be 
strange irregularities or discontinuities in the second derivatives. 

While these characteristics of the surfaces S; may seem plausible for the 
mental abilities, there do not seem to be any similar obvious restrictions for 
the surfaces that represent temperamental traits. If some of these surfaces 
are characterized by conspicuous maxima and minima, then the observa- 
tion equations for them might have to be of the third or higher degree. 
Gross discrepancies and large residuals may then be found in the crude 
linear assumptions on which present-day factorial methods are based. Most 
of the statistical methods in current use involve the same assumption, 

The various rating schedules for personality traits are frequently ar- 
ranged with special reference to social acceptability and success rather than 
with reference to the degree of some underlying trait. Such ratings may 
give trouble in factorial studies if the relation between the amount of some 
underlying trait and the rating on some kind of social acceptability has 
à maximum for an optimal amount of the underlying trait. The assumption 
of linearity between the underlying parameters and the overtly observed 
social acceptability is then violated more seriously than if we were dealing 
with personality descriptions that are monotonic increasing or decreasing 


* If time per unit task is used as а Score, then the score function is decreasing 


monotonie, in which case it can be reversed so that all correlations become positive. This 
applies to abilities and not to all psychological measurements. 
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functions of the underlying parameters. An example may clarify these re- 
strictions. Let us suppose that one of the important underlying parameters 
in describing a man is the rate at which he moves generally. A person with 
a very low value of this parameter would be described as slow, inert, and un- 
responsive. The upper extreme of this parameter would be described as a 
personality that is jerky, scatterbrained, nervous, irritable, and perhaps 
unstable. An optimum value between the extremes is likely to be associated 
with more favorable descriptive categories as regards social acceptability 
and success. The sketches of Figure 2 are drawn to show that a monotonic 
increasing or decreasing surface can usually be fitted better by a linear ap- 


proximation than can a surface with conspicuous maxima or minima. 


Ficure 2 


ave represented two plane surfaces which may be thought 
of as best-fitting planes for the score surfaces S; and S+. The score surface S; 
of Figure 1 represents raw scores. Such a surface also could represent stand- 
ard scores by merely moving the origin to the level of the mean and adopt- 
ing a vertical scale with the standard deviation as a unit of measurement. 
With this translation of the origin and with the stretching factor, we have 
the planes passing through the origin as shown in Figure 3. It is of some 
interest to note, in passing, that the rank of a matrix of raw scores is reduced 
by 1 when the scores are modified to deviation scores about the mean. 

The two vectors J and Kin Figure З аге unit normals to the planes S; and 
Si, respectively. The direction cosines of each normal are also the coeffi- 
cients of the equation of the corresponding test score plane. If a surface Sj is 
cylindrical and parallel to some axis X,, then the best-fitting plane for that 
surface is also parallel to that axis except for the accidents of sampling. 
The direetion cosines of the normal to the plane will have a zero-direction 
cosine for the axis Xp. The vectors J and К are unit test vectors in the total 
test space. If X; and Х» are correlated and if a number of test vectors are 
orthogonal to them so that these axes are chosen for the oblique factorial 
matrix Уу, then the test vectors will have zero projections on these axes 


In Figure 3 weh 
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and the corresponding entries in the factor matrix will be zeros. In this 
manner we see that if a number of test score surfaces S; are cylindrical and 
parallel to each other, then the corresponding axis X, is chosen as a refer- 
ence vector and is represented by a large number of zero entries in a column 
of the oblique factor matrix. 

The relations of Figure 1 may be used to illustrate another fundamental 
principle in factor analysis as regards the scientific interpretation of a simple 
structure when it is found. Suppose that there are individu 
the ability to visualize space. A test battery could be 
visual tests that do not involve this factor 
volve it. The factorial 


al differences in 
assembled, including 
and other visual tests that do in- 
separation would sustain the hypothesis that such a 


Figure 3 


factor is distinguish 


able among visual functions. If a parameter or factor or 
function exists w 


hich becomes operative in some way in the visualizing of 
space and not in other visual tasks, then we should expect those score sur- 
faces to be parallel to an axis which does not involve the particul 
eter. If the separation of the Score surfaces 
fication of the tests as including or exe 


the factorial results sustain the hy 


ar param- 
actually agrees with the classi- 
luding the particular funetion, then 
pothesis. 

The principle of interest here is that the identification of a factor or 
parameter does not presuppose the experimental population to be in any 
Sense representative of any sort of hypothetical general population. In fac- 
torial studies that are made for the discovery of an underlying order in a 
domain, the best. procedure is not to select a random group of subjects, 
but rather to select the subjects so that their attributes are as diverse 
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as possible in the domain to be studied. Two or three freaks in the charac- 
teristics of interest are worth more than fifty average subjects. This prin- 
ciple is not new. If you wanted to inquire informally about the existence of 
the ability to visualize space, you would gain most by selecting some drafts- 
men or engineers and some people who evidently avoid visualizing tasks and 
methods of thinking. Later, when the factor had been identified clearly on 
exceptionally gifted or deficient individuals, you could proceed to the task 
of establishing norms to determine the actual range of ability in random 
samples of the general population. Here, again, we see how the purpose of 
a factorial study determines the kind of subjects to be selected for the ex- 
perimental group. If we have no hypotheses about the kind of people who 
differ fundamentally in the domain to be studied, then we proceed by in- 
viting any conveniently available group of subjects in the hope that they 
will differ markedly among themselves in the several parameters or factors 
of the domain to be investigated. If a few subjects can be found who are 
known to differ widely in the domain, they should be included. No assump- 
tion of normality of the distribution is involved in factorial analysis. If we 
could find bimodal distributions for some tests and factors, they would be 
especially interesting, provided, of course, that the bimodalities were not 
artifacts. The question of whether the correlations between the tests in an 
ation are in any sense representative of a general popu- 
lation is irrelevant. Nor does it matter whether the correlations for one ех- 
perimental group differed markedly from the correlations of the same tests in 
another experimental group. If the factorial analyses are made NERE 
ently, the same factors should be identified, but the vim values would; 
of course, be different if the groups differ widely. The correlations for each 
termined by the factors involved in the tests and by the corre- 
ich experimental group. 
as the field of all possible measurements 
that are det ermined, at least in part, by a set of parameters. In exploratory 
studies the domain is defined tentatively in gui of the content or Nature 
of the tests of a battery. The principal purpose is to discover the param- 
and something about the nature of the individual differences 
: dividual subjects are examined, not for the pur- 
about them individually, but rather for the pur- 
pose of discovering the underlying porn: О be 
able to appraise each individual as to a of the factors, hi$ problem 
raises certain other questions about Vane ; wien ; : ‘ 
А he fundamental linear observation equation for factor 

In setting UP : [score Sji İS expressed in terms of the test coefficients 
analysis, the od arai ша set of arbitrary orthogonal factors tmi. By 
a a a iip aie can be expressed as linear combinations of indi- 
rotation of axes E 7 


experimental popul 


group are de 
lations of the factors 1n € : 
We have defined the domain 


eters or factors Š 

"m 
that they produce. The in 
pose of learning something 
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vidual scores xp: in the primary factors p instead of the arbitrary factors m. 
Some writers have attributed to factor analysis the assumption that all the 
subjects in an experimental group use the same factors in doing a test, but 
such an assumption is not made in factor analysis. Suppose that a task can 
be done in one of two ways that involve either of two factors, 1 and 2. Sup- 
pose, further, that some subjects use factor 1 and that other subjects prefer 
factor 2 in doing this task. If this test is included in a battery for factor 
analysis and if the battery contains other tests by which factors 1 and 2 
can be clearly differentiated, then the test in question will have a complexity 
of at least 2. We should then infer that the test has these two factors, in 
that од and vj» are non-vanishing in the factor matrix V; but it would not 
follow that each individual subject used both of these f. 
test. To decide whether this is true, 
doing the task to ascertain wh 
both factors should be simult: 


actors in doing the 
one turns, first, to introspection in 
ether it seems psychologically necessary that 
aneously used, or whether it is a question for 
each subject to decide which method to adopt in doing the test. Here one 
must be careful not to be biased by one’s own preferred ways of doing 
things. In case of doubt, we ask a number of subjects how they did the task, 
including the instructions and fore-exercise, and to describe any useful 
tricks or dodges that they may have discovered in doing the test. These in- 
formal reports are Sometimes very illuminating. A little experience of this 
kind soon emphasizes the importance of introspection as compared with 
casual judgments about the factorial composition of a test merely by a su- 


r example, a syllogism test about the 
as found to have a significant saturation on 
although the test was entirely verbal in appearance. It 
was found that some subjects, but not all, used simple diagrammatic repre- 
sentation of the premises such as “S—J” to represent the premise that 
Smith is taller than Jones, so that the inference was also spatially repre- 
sented. In order to make the analysis more complete, one might separate 
the subjects into two groups according to preferred methods of doing a test 
and reanalyze the results, In such a situation we should expect to find a 
different factorial composition of a test for the two groups of subjects. 
Another situation in which the factorial composition of a test can be 
altered is that in which the subjects are given instruction in doing a task. 
The factorial composition can be markedly changed, since the test is itself 


L 
what goes on in the mind of the subje 


test paper. In the same manner the f 
with the age of the subjects. Somet 
tests in a battery with reference to 


actorial composition of a test is changed 
Imes we have arranged the order of the 
the suggestive effects of one test on a 
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following test, especially if one test requires a mental set similar, or oppo- 
site, to that of a previous test. Fortunately, these overlapping effects are 
not often serious. 

When it comes to the problem of using the tests, not for the purpose of 
identifying mental factors, but for the purpose of appraising the factors in 
each individual, we resort to the same methods that are in current use with 
psychological tests. The best that we сап do is to determine the correlation 
of each test with each primary factor, гур. These are the validity coefficients 
of the tests with respect to the primary factors. We select several tests 
whose validities are the highest available for a particular primary factor 
p. These several tests are pooled in a composite. The composite score is 
then used as the best available estimate of the individual's score ть in the 
primary factor p. If each test has saturations in several factors, such as 
(1, 2), (1,3, 5), (1,4, 7), then we try to distribute the distracting saturations 
for other factors so as to emphasize the validity for one of the factors. The 
same result can be obtained by multiple correlation analysis in the usual 
ways where rjp аге validity coefficients which should be studied with the 


intercorrelations of the separate tests. 


Simple structure in 7 dimensions 
ture principle will be restated here for r dimensions. 


Some of the previously given definitions will be repeated in this context.* 
The r orthogonal reference vectors that are implied in the orthogonal fac- 
tor matrix F are called orthogonal reference vectors. These vectors are re- 
garded as a fixed orthogonal frame in terms of which the various factorial 
relations are expressed. It is this fixed frame in terms of which we describe 
the test vectors j in the rows of Р, the r hyperplanes of an orthogonal or 
oblique simple structure, and the primary trait vectors. T hese fixed orthog- 
are referred to by the subscript m, or M and m if two subseripts 
are desired. Thus the test vector j has the projections аы ON these. orthog- 
n in the rows of F. When this reference frame is deter- 
he centroid method, the frame will be called centroid 


The simple-struc 


onal axes 


onal axes, as Show 
mined by some form of t 


A aun configuration of trait vectors defined by the correlation ma- 
trix is called the correlational configuration or the trait configuration. This 
is the configuration described in terms of the fixed orthogonal frame by the 
factor matrix F. The combination of the correlational configuration and 
any set of reference vectors 18 called a structure, and it can be seen that a 
structure is itself a configuration of (n+r) vectors. The problem is to sub- 

frame another frame, orthogonal or 


stitute for the arbitrary reference 1 : 
ДЇЇ have scientific interpretation. Each reference vector of 


oblique, which w 


* First ed., chap. vi. 
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the new frame should represent a scientific category, a parameter, or a 
function in terms of which the trait configuration can be interpreted. 

The r linearly independent reference vectors define the same number of 
co-ordinate hyperplanes. Each hyperplane is a subspace of dimensionality 
(r—1), and it is defined by a reference vector. The subspace of (r—1) di- 
mensions which is orthogonal to the reference vector A, will be called the 
co-ordinate hyperplane Lp. If a reference frame can be found such that cach 
test vector is contained in one or more of the r co-ordinate hyperplanes, then 
the combined frame and configuration is called a simple structure. If the 
new reference vectors are orthogonal, the total configuration is called an 
orthogonal simple structure. If a set of r hyperplanes of dimensionality (r—1) 
exists such that each test vector is in one or more of the hyperplanes and if 
the normals A, are oblique, then the total configuration of test vectors and 
reference vectors is called an oblique simple structure. It follows from these 
definitions that each test vector that contributes to the identification of a 
simple structure must be of complexity less than r. A configuration of test 
vectors which defines a simple structure will be called a simple configuration. 

If an nXr factorial matrix is set up with arbitrary entries in all cells, 
there is, in general, no transformation by which each of the » variables can 
be described in terms of fewer than r factors, It is assumed that n is large 
in comparison with r, Therefore, the appearance of simple structure in a 
factorial matrix derived from observation commands attention. It is not ü 
chance matter, When found in experimental data, it reveals order within 
the n variables, in that 7 «n categories are required for describing them col- 
lectively and fewer than r categories are required for each one of them 
Separately, 


If an underlying 
traits can be d 


physical order of the n traits is such that each of the 
escribed in terms of a smaller number of factors than are re- 
quired for describing the traits collectively, then the underlying physical 
order will be called a simple order. 

If a simple order exists for a set of n traits and if the r factors are statis- 
tically independent in the experimenta] population, then the corresponding 
physical order will be called an orthogonal simple order. Hence the configu- 
ration which represents an orthogonal simple order among the n traits is an 
orthogonal simple Structure, An order among the traits involves, of course, 
not only the traits themselves but also the categories in terms of which they 
are described, These categories are themselves traits which may or may not 
be experimentally isolable. 

It is useful to summarize the several fundamental concepts in this anal- 
ysis. The concept order refers to the relation bet ween the traits and the 
categories in terms of which the traits are to be described and compre- 


hended. The correlational matrix describes merely the relations among the 
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traits, independent of the descriptive categories. The factorial matrix de- 
seribes the traits or variables in terms of a set of descriptive categories. The 
trait configuration is a geometrical representation of the correlational ma- 
trix, and hence it is also independent of the descriptive categories. The 
structure is a configuration which represents not only the traits but also the 
descriptive categories. The scientific problem is essentially a search for a 
set of descriptive categories in terms of which our conception of the traits 
or variables is the simplest possible. If an overdetermined simplicity in our 
conception ean be achieved, then the traits or variables will be said to re- 
veal a simple order. The search for these categories has its direct analytical 
counterpart in the search for a set of reference vectors which will reveal a 
simple structure. A simple structure is a configurational representation of a 
simple order. If the simplifying descriptive categories happen to be statis- 
tically independent in the experimental population, then the trait configura- 
tion can be so rotated in its arbitrary orthogonal frame that each trait vec- 
tor is contained in one or more of the r orthogonal co-ordinate hyperplanes. 
The result is an orthogonal simple structure, and the reference vectors repre- 
sent a set of statistically independent traits that serve the simplest possible 
comprehension of the given traits or variables. 


Alternative simple structures for the same domain 


We should distinguish between uniqueness of the simple structure for a 
given correlation matrix and uniqueness of the descriptive parameters or 
concepts which the primary axes represent. In earlier chapters we have con- 
sidered examples, such as the cylinder problem and the box problem, in 
which the simple structure does not give the only meaningful set of param- 
eters or descriptive concepts for a configuration. It is unlikely that unique- 
ness in that sense could be found in any domain. Furthermore, it is con- 
ceivable that a domain could be represented by two sets of measures which 
would show two different simple structures, if the two sets of measures were 
different ways. Both sets of primary axes would then repre- 
ameters for which meaningful interpretation could be 
expected. The two sets of parameters could be regarded as alternative sets, 
and they should be revealing as to the nature of the underlying order. 

The box problem, which can be easily visualized, has served to illustrate 
these principles. Consider one set of measurements of the boxes witch are 
simple functions of the three independent edges, as in the previous examples. 
Consider also another set of conceivable measures of the same populat ion of 
boxes in which each measure is some simple function of the basic parameters 

ios ё=т/г and u=2x/y. The three measures р, t, 


for volume, v, and the rati east 
and и are superimposed on the simple structure хт, y, and 2 in Figure 4. 


рар represent width, height, s ` 
The basic measures т, Y and z might represent width, height, and depth, 


taken in quite 
sent underlying par 
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respectively. It is instructive to see what would happen with different sets 
of measurements of the boxes. First, consider the case in which we merely 
add the three measures v, t, and u to the measures previously listed and 
which contribute to the simple structure of the box problem. The result 
would be merely the same triangular arrangement of the measures forming 
the triangle x, y, and z, and there would be the three additional measures 
v, и, and t, which would not contribute to the identification of the simple 
structure. In the first example of the box problem we had two such meas- 
ures, namely, 16 and 17, which represented the volume and the principal 
diagonal of each box. These did not participate in the simple structure. 


x. 


2 


FIGURE 4 


If we should construct an 


entirely new set of measurements which were 
so composed that they wer 


€ linear combinations of the three measures 
v, t, and u, then they would define a new simple structure with primary vec- 


tors v, 1, and u. The previous configuration z, y, and z would not be seen. 
If we proceeded to construct this т 


measures such as “one-half of the 
of the ratio z/z" and perhaps t 
Should then be able to show th 
unique for the domain. The respi 
depend entirely on the 
tained. If the new stru 
linearly independent meas 


attery, then, of 
ure would be regarded as an artifact, and it would 


e domain. If an 


course, the new struct 
demonstrate nothing г 
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author were caught doing such a thing, his scientific honesty would be 
questioned unless he presented it merely as some sort of stunt. The arbi- 
trary character of his battery of measurements would be questioned. 

But there could be situations in which the new structure v, t, u would 
be valid and signifieant. Suppose that these new parameters were signifi- 
cantly related to the manner in which the box shapes were generated or in 
which they were used or interpreted. Then an investigator who assembled 
a battery of measurements to represent the domain might find the structure 
v, 1, u, and it would serve him in studying the underlying order. The as- 
sembled battery would reveal its organization in the new structure. 

Let us suppose that two investigators found these two simple structures, 
using two different batteries of box measurements. Each of them might find 
a set of primary factors, and each might succeed in discovering some 
appropriate interpretation for his own set of three basic parameters. 
Such a finding would not necessarily show that one of them was wrong 
They might have found two sets of alternative parameters for describ- 
ing different aspects of the domain in question. The challenge is then 
to study closely the two sets of measures to sce what different aspects 
of the domain they cover. A new factorial experiment could be set up 
with both sets of measures in the same battery, and the configuration 
would then be like Figure 4. Additional factors might appear because 
the previous specifies might appear as common factors in the new analy- 
sis; but our present concern is about the principle involved in the possi- 
bility of alternative structures. This would be a case in which there would 
be more hyperplanes defined by the test vectors than there are dimen- 
sions in the common-factor space. Such a situation should not be regarded 
as a failure in factorial analysis. It might reveal alternative sets of param- 
eters in the domain, which should all eventually be interpreted. So far we 
have found only an occasional suggestion of secondary hyperplanes, but 
none of them have seemed to us to be sufficiently overdetermined to justify 
special investigation. 

n Tem m difference between alternative structures that should be 
separately interpreted and the artifact which could be arbitrarily con- 
strueted from any test battery, it is well to look at the nature of the meas- 
urements involved. Both sets of measurements should be natural and ap- 
parently relevant measures descriptive of the domain in question. If one 
set of measurements consists merely of a set of linear combinations of & 
few previous measures, or the equivalent as to arbitrary character, then its 
relevance to the problem of analyzing the domain may be questioned. In 
any event, subsequent factorial study of the same domain should reveal 


whether a previously found structure is to be sustained. 
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Uniqueness of primary factors 


One of the most fundamental problems in factor analysis concerns the 
uniqueness of the primary factors and their interpretation. When a factor 
analysis has been completed with a clear simple structure, the first question 
is naturally: “What are the parameters or factors?" It seems to be in the 
nature of science that such a question has no unique answer, In scientific 
work a parameter is one of the measurable attributes of an object in terms of 
which it is described. In general, one tries to choose a small number of 
parameters which define all the relevant attributes of the object. The au- 
thor is successful if he finds a small number of easily comprehended param- 
eters that cover his problem, Parameters in scientifie work and the con- 
cepts which they represent become the conventionalized ch 
in textbooks. Even notation becomes a matt 
a graph is defined by the two co-ordinate: 
radius r. These parameters are easily 
problems, but, of course, they 
would be awkward, a 
same is true in 


apter headings 
er of convention. A circle on 
s (x, y) of its center and by the 
understood and easily used in most. 
are not unique. For some problems they 
ind another set of parameters would be chosen. The 
scientific work. The familiarity of mass 
acceleration should not lead us to suppose th 
for the description of objects in motion, 
uniqueness of parameters apply 
acknowledge them also in the 
al unities of mind, 

The choice of a set of fund 
main of nature is to be с 
The choice is for 


‚ momentum, and 
at they are necessary absolutes 
If these reservations about the 
to physical science, it would seem wise to 
search for descriptive parameters or function- 


amental concepts in terms of which any do- 
omprehended 15 probably meaningless to nature. 
us to make in terms that will attain intellectual control and 
consistency, as empirically determined, and by which this control сап be 
attained, as far as Possible, with ideas that are already familiar to us, Fun- 
damental scientific concepts that are successful in this sense are as social as 
other inventions. To hunt for a unique solution in the comprehension of a 
set of related phenomena is an illusory hunt for absolutes. It is probably 
safe to say that an apparently unique set of concepts in any domain is mere- 
ly the symbol of our immaturity in the exploration of that domain, The 
recognition of the lack of uniqueness in scientific concepts does not imply 
that some sets are not more useful and fruitful than others, Those param- 
eters are preferred which reveal the Phenomena as of 
order. 

Recognition of the fact that descriptive parameters 
led some writers on factor anal 
remain content with a set of st 
which are wholly determine 
moment. The simple-struc 
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can be called an “invention,” as a compromise with this problem. It was 
the simple idea of finding the smallest number of parameters for describing 
each test. In numerical terms this is a demand for the smallest number of 
non-vanishing entries in each row of the oblique factor matrix. It seems 
strange indeed, and it was entirely unexpected, that so simple and plausible 
an idea should meet with a storm of protest from the statisticians. This 
simple idea has turned out to be a much more powerful analytical device 
than was at first anticipated. 3 j K 
The primary factors that are identified by the simple-structure principle 
seem to be unique in the sense that they reappear in successive studies with 
different test batteries in the same domain. It is to be expected, however, 
that a primary factor may be found to be itself a complex when a part of a 
domain is investigated with large test batteries. It will then give ps 
Several new primary factors, which should retain their identity in succ essive 
test batteries that are adequate to isolate them. As regards [genesis 
interpretation, it may be expected that this will а be ee or л е 
with repeated studies. The number factor, for E E eine 50 
far only in terms of the numerical content of the б; ш p ve 0 
terpretation cannot be regarded as satisfactory. M ie не а е ан 
pretation should be made eventually in terms 0 oa » ogie a Pte: ‘i 
that will generally transcend test content excepi " гнет Кя ues 
test content and the psychological processes a еа трзд бананов 
name, In order for the interpretation of ЫШ к ei Din A i à 
their psychological interpretation should La ep dein D es 
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cilitated by examining their genetic basis and by noting the order and the 
rate of development of the different functions. Thus, for example, the 
ability to do the Street Gestalt Completion test seems to be a function 
that matures very early, perhaps as early as six or seven years, while the 
verbal factor seems to show continued growth through the high-school 
years. The growth of the verbal factor may be explained by the imperfec- 
tion of present tests, which are markedly influenced by schooling and ex- 
perience or by slow maturation of the ability itself. At any rate, the primary 
factors which are identified by the study of individual differences at point 
age should be related to other findings on the rate of growth of the mental 
functions. The primary factors should preferably be a set of parameters 
which can be identified at each age by appropriate tests. The acceptance of 
primary factors in psychological investigation will depend on the extent. 
to which they are fruitful in non-factorial psychological research. If they 
represent truly significant parameters for the description of mental traits, 
then they should also be revealed as distinguishable in the study of mental 
inheritance. It is to be hoped that at least some of the primary factors that 
are now being identified in the study of individual differences at point age 
will also be found to be useful and simplifying parameters in the study of 
mental inheritance and in the study of mental growth. 


Uniqueness of simple structure in a given correlation matrix 


{ When reference axes have been found which produce a simple structure, 
it is of considerable scientific interest to know whether the simple structure 
is unique for the given correlation matrix. The necessary and sufficient 
conditions for uniqueness of a simple structure need to be investigated. In 


the absence of а, complete solution to this problem, five criteria will here be 


listed which probably constitute sufficient conditions for the uniqueness of a 
simple structure. The ge 


bli ‘a : ientific interpretation of the cell entries in the 
9 ae E matrix V should not be attempted except after reasonable 
assurance that the simple structure of V is unique for the given configura- 


tion. If this assurance i avai ; R 
n. If this assurance is not available, then the interpretation should be 


tentative. 
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need only eliminate from consideration those tesis in the oblique factor 
matrix which have complexity 7, because the other eliminations are made 
routinely in factoring the correlation matrix to save the unnecessary labor 
of carrying some tests through the factoring when they cannot contribute 
to the analysis. 

If we eliminate the tests of complexity 7, the remaining tests should con- 
tribute to the location of the reference frame. We shall describe five useful 
criteria by which the r reference vectors can be determined. These are as 


follows: 

1) Each row of the oblique factor matrix V should have at least one zero. 

2) For each column p of the factor matrix V there should be a distinct 
set of r linearly independent tests whose factor loadings vj; are zero. 

3) For every pair of columns of V there should be several tests whose 
entries 0j, vanish in one column but not in the other. 

4) For every pair of columns of V, a large proportion of the tests should 
have zero entries in both columns. This applies to factor problems with 
four or five or more common factors. 

5) For every pair of columns there should preferably be only a small 

. number of tests with non-vanishing entries in both columns. 

When these conditions are satisfied, the plot of each pair of columns 
shows (1) a large concentration of points in two radial streaks, (2) a large 
number of points at or near the origin, and (3) only a small number of 
points off the two radial streaks. For a configuration of r dimensions there 
are ir(r—1) diagrams. When all of them satisfy the three characteristics, 
we say that the structure is "compelling," and we have good assurance 
that the simple structure is unique. In the last analysis it is the appearance 
of the diagrams that determines, more than any other criterion, which of 
the hyperplanes of the simple structure are convincing and whether the 


whole configuration is to be accepted as stable and ready for interpreta- 


tion.* ) 
The first criterion demands that each trait should be describable in terms 


of fewer categories than are required by the whole set of n traits. It is con- 
ceivable that, in some experimental work, one or more of the traits will be 
so complex as to require description in terms of all the factors that enter into 
the traits collectively. For the purpose of isolating the fundamental cate- 
gories, these traits are not useful, and they should therefore be ignored. 
The criterion demands that the list of traits be long enough so that, after 
elimination of several traits of complexity r, enough traits of complexity 
less than r remain to determine uniquely both the trait configuration and 
the simple structure. This principle may be illustrated with psychological 
simple-structure solution for the factor problem, I have never 


ver si und the i ; 
* Ever since 1 fo al result without first inspecting the diagrams. 


attempted interpretation of a factori: 
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tests. If one of the abilities to be isolated should be number sense, then this 
primary ability should not be required in all the tests of a battery. The 
same restriction applies to each of the abilities that is to be isolated. 

The second criterion seems to be essential for the following reason. Each 
column p of V is determined by a hyperplane Lp. A hyperplane through 
the origin is determined by (r—1) trait vectors. These trait vectors are 
contained in L,, and therefore they have v 


anishing entries v; in column p. 
Therefore, there must be at least ( 


r—1) traits with vanishing entries in 
each column of V in order that the hyperplanes may be determined. Since 
the hyperplanes should be overdetermined by the data, it follows that the 
number of vanishing entries in each colur 

The third criterion is suggested by the fact that the r hyperplanes must 
be distinct. If two columns of V contain the same vanishing entries and if 


these exceed (r—2) in number, then, assuming the test vectors to be linearly 
independent, the two correspondin 
criterion was w 


nn of V should equal or exceed r. 


g hyperplanes are identical. The third 
ritten so as to insure both overdetermination and distinct- 
ness of the hyperplanes that define the columns of V. 

When the fourth criterion is not satisfied, i.e., when there are no points 
at or near the origin for a problem of more than four or five dimensions, we 
ыр be reasonably sure that the hyperplanes have not been correctly located. 
This has been our experience with large and small factor problems. The 
fourth criterion is of importance in factor problems of four or more dimen- 
pons: In a three-dimensional problem a point at the origin on the diagram 
for factors 1 and 2 would have to be a measure of only the third common fac- 
bas so that its complexity would be 1. In two dimensions there can be no 
points at the origin except for a null vector, Such a vector is not found in a 


factor problem because its correlations would all be zero or small so that it 
would be eliminated from the te 


Е st battery before factoring was begun. 
The problem of determining t : 


3 $ 1e uniqueness of a simple structure may not 
be во serious as it seems at first sight. We are accustomed to the demand of 
statistical methods that a solution shall be unique in order to be classified 
as a preferred method. Actually, we do not often attain uniqueness. The 
curve-fitting methods in general use are not unique even when the same 
methods are used, as, for example, when a least-squares solution gives sev- 
eral variant answers, depending on which of the variables the investigator 
chooses to be dependent on the rest of the variables. However, it would be 
very useful to have a solution to the simple-structure problem by which we 
could know by some explicit method whether or not a frame exists in the 
configuration that gives a better fit than the one we may have found. 

In presenting a simple structure аз a solution to a factor problem, several 
considerations determine the plausibility and acceptability of the solution. 
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The plausibility of a solution should be considered separately for each fac- 
tor. Some of the relevant criteria will be mentioned briefly: 

1) The degree of ambiguity or plausibility of each hyperplane as judged 
by the configuration. In Figure 5 we have illustrated for a three- 
dimensional configuration how the several co-ordinate planes frequently 
differ widely in degree of plausibility and convincingness. The plane L is 
overdetermined by more than enough points (test vectors) to define the 
plane, and hence the existence of a parameter T, seems clearly indicated. 
The plane Ls is barely determined, since two points are the least that will 
define the plane in three dimensions. The existence of the parameter T; is 
suggested but not demonstrated. The plane Ls is indeterminate, as shown 
by several possible locations in the dotted lines. The parameter T» is sug- 
gested but not demonstrated. In Figure 6 we have a three-dimensional con- 
figuration of six test vectors, in which the structure is not at all convincing. 


FIGURE 5 FIGURE 6 


The sets of solid lines and dotted lines show two entirely different simple 
structures, neither of which is overdetermined. We see here how it is not 
likely that some single criterion will be found for the uniqueness of a simple 
structure, because the several planes and their corresponding parameters 
may differ widely in their = of overdetermination and convincingness 
in any particular factor study. 
zi a a paul of the theoretical interpretation that the investigator 
may be able to find for each hyperplane. When an unexpected plane ap- 
pears in a faetorial study, it is a challenge to find an interpretation for ih. 
If the interpretation seems plausible and if it clearly divides the tests into 
two categories, those with strong non-vanishing values of Vj, and those 
ith negligible factor loadings, the interpretation сап be accepted at least 
T. thesis worthy of further study. This task of interpreting the fac- 
s үр - н carly the most important part of a factorial study. If the 
жарыл o te TOKEY, then so is the whole study, no matter how elegant 
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the statistical work may have been. Exploratory studies in a new domain 
are likely to give clues only as to the nature of the underlying functions or 
parameters. р 

3) The predictability of a postulated factor. In order to set up a crucial 
test of the word fluency factor W, two new tests of Synonyms were con- 
structed, one of which was predicted to have high saturation on W and no 
saturation on the verbal factor V, while the other Synonyms test was pre- 
dicted to behave in just the opposite way, namely, zero on word fluency 
W and high on the verbal factor V. They did behave in the way predieted 
in the next factorial study. This did not prove the interpretation of W to be 
finally correct, but it did sustain the hypothesis concerning the differentia- 
tion of V and W. Eventually, the factor W may be redefined by other 
studies so that it might even transcend the verbal content altogether. 
Here, as in other experimentation, success depends very largely on how suc- 
cessful we may be in inventing tests and measurements that will be crucial 
in differentiating between rival hypotheses concerning the nature of the 
factors or in testing a tentative interpretation indicated by previous studies, 
It may happen that one experimenter will find a factor, that another will 
interpret it with some hypothesis, and that a third will invent the crucial 
tests by which the hypothesis can be examined factorially or otherwise. 
It is in this manner that factor analysis can find an important role as a 
scientific method not only in psychology but in other sciences in which ex- 
ploratory studies may reveal an underlying order in a new domain, 

In making interpretations for a new factor that has been identified fac- 
torially, we distinguish, in the spontaneity of our approval, between inter- 
pretations that seem vague or far-fetched and those occasional incisive in- 
terpretations that “Click,” as it were, and by which the factorial results 
snap into a meaningful whole, This is only to say that the scientific inter- 
pretation of experiments is as subjective now as it alws 
mand that factorial interpretation shall be objective is 
mand that the interpret 
It never is. 


ys has been. To de- 
as absurd as to de- 
ation of a physical experiment shall be objective. 


4) The degree of convincingness of 
described as due, in part, to the overdet 
this judgment is made also with due re 
vectors that do not contribute to defin 
we have drawn two plots that n 
faetor matrix, In the first. we 
configuration with perhaps e 
it would not be convincing, Р feel quite con- 
vinced that an underlying order is rey, 
points makes the location of t 


he two axes quite compelling, Here our judg- 
ment is influenced by the rel 


ative number of points that do not contribute 
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to the definition of the simple structure. In the second plot we have drawn 
several points in the middle of the first quadrant. If an interpretation can 
be found for the factors A and B, then it is necessary to explain how it 
comes about that the several tests in the first quadrant have some satura- 
tion in both A and В. If such an explanation seems plausible, it strengthens 
the interpretation of the factors. A configuration such as the first in Fig- 
ure 7 leaves one unconvinced, no matter where the axes are drawn, unless 
an interpetation can be found that seems right. Random configurations like 
this seldom yield clear interpretations, but they are not, of course, physical- 


ly impossible. 


FIGURE 7 


Some critics have pointed out that if the factorial composition is known 
for a large test battery, it would be possible to pick and choose a set of 
tests from the larger battery so that they would reveal a simple structure 
even for some more or less arbitrarily selected reference frame and that the 
simple structure so put together would not demonstrate any validity for 
the arbitrary reference frame. This is entirely correct. This is only one of 
many ways in which a dishonest author can fake scientific data. Another 
well-known сазе is that in which one can pick and choose points in a corre- 
lation table, erasing the points in the second and fourth quadrants, so as to 
change a zero correlation to a high positive value. If a test battery is as- 
sembled with tests that are crucial for one or several rival hypotheses, the 
configuration of the whole battery should be used in the interpretation. If 
the interpretation is valid, it should be possible to construct new and differ- 
ent tests with prediction as to how they should behave factorially. Again 
we should want to see the hypotheses, the tests, and the whole configuration. 

Another criticism of simple structure which is in the same class is that 
one gets factorially only what one puts into a test battery and hence that 
the separate factors have no validity and represent only the arbitrary cate- 
gories imposed by an author. It is a platitude to say that one gets out of a 
test battery only what one puts into it. Again, that is true for any scientific 
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experiment. When a test battery is constructed with some hypothesis 
which is to be examined, it does not follow that the factorial analysis neces- 
sarily supports the categories that guided the construction of the battery. 
Most of our disappointments in factorial studies are, in fact, exactly of this 
kind. Hence we can say on the basis of considerable experience that a factor 
analysis does not necessarily reproduce the categories that constitute the 
preconceptions of the investigator. Any factors that are found are certainly 
represented by the tests that were put into the battery, but they do not 
necessarily fit the preconceptions of the experimenter. 

The various criteria that have been discussed here seem obvious enough 
when we deal with graphical methods of inspecting the configuration, but 
they are likely to prove troublesome for anyone who attempts to reduce 
them all in some algebraic manner to a single criterion of uniqueness of 
simple structure. Any contributions in this direction are likely to be helpful 
in dealing with partial aspects of the problem of finding the most plausible 
reference frame for any given configuration. It must not be forgotten that 
it is an open question for each factorial study whether any simple structure 
exists at all or whether any one or several hyperplanes can be defined. It 
should also be noted that a factor study may be a major scientific contribu- 
tion if it reveals only a single clear factor which can be given a fruitful and 
provocative interpretation, even if all the other factors in the same study 
prove to be entirely indeterminate. Furthermore, such a factorial result 
might be of fundamental significance, even if, say, only a fourth or a third 
of the variance of the crucial tests is accounted for by the factor in question. 
Other things being equal, a factor will be the more convincing, the higher 
the saturations involved in the tests that involve the factor. But the exist- 
ence of large parts of the variance as unique or as uninterpreted common 
factors does not preclude Significant findings. 

Factorial studies may be roughly classified as of two kinds. When there 
are no hypotheses to guide the study of a new domain, the experimenter 
tries to cover the domain with tests or measures that are diversified within 
the domain. The factorial result is likely to give an incomplete simple struc- 
ture at best, but the exploratory study is justified if one or more functions 
or parameters appear in the interpretation that can be used as starting- 
points for more crucial subsequent studies. The second kind is the experi- 
ment in which some hypotheses are available and in which tests are specially 
constructed to represent the presence and the absence of each postulated fac- 


tor or parameter. Such studies are likely to be more productive of definitive 
results. 


Interpretation of zero factor loadings 


AR E n А 
There 18 occasional misunderst 


anding about the interpretation of zero 
factor loadings. A zero f. 


actor loading in a test is sometimes spoken of as 
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if it meant the absence of the factor in the test, but that is not thi 

interpretation. A zero factor loading means that the factor in TT 
does not contribute to the variance of the test. The difference can e 
an example. Consider two hypothetical factors, spelling and writin т 4 
If a simple spelling test were given by the time-limit method to А rani f 
educated adults, the number of right answers might represent desi 
speed of writing, because the educated adults could be assumed to e vm 
to spell all the simple words. Hence the variance of this test would da " 
tributable entirely to writing speed and none to spelling abilit; hk 
would be incorrect, strictly speaking, to say that the test does Su ле 
any spelling ability. If the subjects were all equally good in a factor or if 
they all reach the ceiling of the test as regards power, the factor would not 
be responsible for any individual differences in the test, and the factor 
would not be identified. i oer 

If a spelling test of greater difficulty were given to the same population 
with ample writing time, then the factorial composition would be markedly 
changed. The greater part of the variance might then be attributable d 
the spelling factor, while the loading on writing speed would be zero The 
zero loading on writing speed would be caused by the fact that this faction 
contributed nothing to the variance of the test, even though all the sub- 
jects actually wrote the test. 

In the interpretation of zero factor loadings we sometimes say that the 
factor is not involved in the test when we really should say that the factor 
is not involved in the test variance. Since we deal mostly with common fac- 
tors, we speak of the factors in a test when common factors are implika, 


The positive manifold 
The principle of simple structure is frequently described as if it assumed 


all factor loadings to be positive or zero. That is perhaps the most common 
misunderstanding about the principle. The reason is probably that the fac- 
torial methods have been applied more in the field of mental abilities than 
in any other field, and it is here that test correlations have been found uni- 
versally to be positive or zero. It is therefore natural to postulate that 
when a unique simple structure is found for a battery of tests of mental 
abilities, then the non-vanishing entries in the factorial matrix are positive 
But it must be understood that this is a postulate of much psychologii 
investigation in a particular field and that it does not constitute a меат 
upon the principle of simple structure, which is applicable either with or 
without the special restriction that the factor loadings shall be positive or 
zero. 

The scientific problems to which the factor methods are applied may re- 
at restrictions on the elements of the factorial matrix. Some 


quire differer 
ictions may be considered under four cases, as follows: 


of these restr 
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1) The simplest case is that in which the factorial matrix F can be used 
as determined by the centroid method, or by any other equivalent method, 
without restrictions beyond those that are inherent in F. It is probably 
seldom that a scientific problem can be adequately solved without some re- 
strictions on the elements of F. 

2) One form of constraint that is of very general scientific interest as re- 
gards the factorial matrix is that of simple structure. 

3) If the scientific problem is such that negative cell entries in F are ex- 
cluded, then we have the important case of a simple structure in which 
аһ Z0. This is the assumption that underlies the application of factorial 
methods to the problem of isolating primary mental abilities; but the as- 
sumption is not absolutely necessary, since ideal constructs can be devised 
for a science of psychology which do not require that the cell entries of F, 
or those of the oblique factorial matrix V, be positive or zero. 

4) A special case of the positive entries of Ё is the further restriction that 
each factor of or of V be either completely present or completely absent 
in each test. This is a case of possible interest in genetics, but it is not likely 


that it will be directly applicable to scientific data without admitting a spe- 
cific variance for each variable. 


If all the elements of V 


are positive or zero, then each column of V is de- 
fined by a positive hy 


perplane so located that all the trait vectors which are 
not contained in it are on the same side of it. If it is assumed that all the 
factors have positive or zero contributions to each variable, then all the 


trait vectors are in the Positive region. The bounding planes of this region 
are then of special interest, 


It will be convenient to п 
vector has only positiv. 
radial vector has only 
region. 

If all the trait vectors th 
of it, the hyperplane will b 
trait configuration. 


If, in addition, a set of r positive hyperplanes exists such that each trait 
vector is contained in one or more of them, then the combined configura- 


tion of the trait vectors and the reference vectors will be called a positive 
simple structure. 


ame the bounded space within which any radial 
€ direction cosines. The bounded space in which any 
positive direction cosines will be called the positive 


at do not lie in a hyperplane are on the same side 
е called a positive hyperplane with reference to the 


ue of vj, in V in the case of mental abilit 
lie in the positive region of the common. 
is satisfied for orthogonal vectors, all th 
or zero. Itisa universally accepted f. 
tal abilities are positive. 


The converse is not necessarily valid. The well-known fact that all inter- 
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test correlations are positive implies that all the test vectors lie inside a cone 
with center at the origin and with a generating angle of 7/4. Such a cone 
cannot be inscribed in the positive region except when the number of di- 
mensions is as low as two. 

The restriction that all the test vectors shall be in the positive region of 
the common-factor space is not sufficient to determine uniquely. In gen- 
eral, there exists an infinite number of orthogonal transformations by which 
all the entries in 7 become positive or zero if the configuration of 7 can be 
inscribed in the positive region. Special cases may be set up in which one, 
and only one, orthogonal transformation will make the entries aj, in F posi- 
tive or zero. Such a case in three dimensions is that in which three test vec- 
tors are mutually orthogonal. These cases are not likely to be found in 
practice. Hence a unique matrix F is not to be expected with the single cri- 
terion that aj, Z0 in F. 

The orthogonal hyperplanes which bound the positive region in r dimen- 
sions will be called the orthogonal positive manifold. 

A set of r distinct and oblique positive hyperplanes for a trait configura- 
tion in r dimensions will be called an oblique positive manifold. 

If the factor matrix of the traits which are contained in a positive hyper- 


plane is of rank (r—1), then the hyperplane is a bounding hyperplane or a 


positive co-ordinate hyperplane. 
The identification of a positive simple structure is done by noting the 


bounding hyperplanes if they are found in the rotations. If a positive mani- 
fold is not postulated or suspected in the test battery, the rotations are 
wd linear concentrations of test points in the diagrams without 


made tow: 
question of whether the hyperplanes are bounding planes. 


reference to the 


Unitary factors 
A special case of the positive manifold is that in which each factor is 


either completely present or entirely absent in each member of the experi- 
mental population. Each individual member of the population has, then, 
one of only two possible standard scores—one positive, which represents the 
presence of the trait, and the other negative, which represents the complete 
absence of the trait. The numerical values of these two possible standard 
scores are determined by the proportion of the population that has the 
trait. If it is assumed in first approximation that the unitary factors are 
equally weighted, then the correlations take only certain possible values 
which are determined by the number of the unitary factors or elements in 
each of the test variables. By inspecting the experimental correlations, one 
can make estimates of the number of elements in each test variable. This 
method of factor analysis seems to be limited in practical application be- 
cause of the serious effects of unique variance and because the elements are 


not likely to be equally weighted in each test variable. However, the restric- 
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tion of the correlation coefficients to certain values makes this principle of 
some theoretical interest, and it might be adapted to factorial studies of cer- 
tain types. It can be regarded as a different type of restriction on the fac- 
torial composition by which the reference axes may be located. This method 
has not yet found practical application.* 


Constellations 


In formulating hypotheses concerning the nature of the primary traits, 
it is sometimes a considerable aid to know of constellations that may exist 
in the trait configuration. By a constellation is meant a grouping of trait 
vectors. It happens not infrequently 
essentially of groups of trait vectors. The angular separations between the 
trait vectors within a constellation are relatively small, while the separa- 
tions between constellations are marked. 

When the dimensionality of the factorial matrix is less than four, the con- 
stellations may be inspected readily by graphical methods. When the di- 
mensionality exceeds three, the graphical methods are not available, and it 
is then useful to have a routine by which the constellations may be isolated 
in the trait configuration. Since the constellations are to be used as an aid 
to intuition regarding the nature of the primary traits, it is not advisable to 
define a constellation rigorously as regards maximum angular separations 
or as regards the maximum generating angle of the cone which will include 
a constellation. Such restrictions may be arbitrarily imposed by the in- 
vestigator for each study. 

If an attempt is made to isolate constell 
traits—say, fifty or more—w 
found that the groupings bec 


that the trait configuration consists 


ations from a large battery of 
ithout some systematic procedure, it is usually 
ome entangled in annoying complexity. If the 
constellations do not exist, the procedure must make this fact evident ; but, 
on the other hand, constellations can be drawn for the purposes of studying 
the battery, even though the traits arrange themselves more in the nature 
of chains than of constellations. In three dimensions this situation is illus- 
trated by a battery of traits whose configuration reveals a spherical triangle, 
in which the sides of the triangle are well defined by the trait vectors. If all 
of them lie in the sides of a spherical triangle, then the isolation of constella- 
tions would be difficult, because there may be no sharp break between one 
constellation and the next. In three dimensions the graphical methods 
would, of course, be used because of their simplicity and directness; but in 
higher dimensions the groupings may be obtained by inspectional methods 
from the intercorrelations corrected for uniqueness. ў 


* See first ed., pp. 205-12, T See first ed., p. 174. 


the factorial methods that have been de Г ade. The inspection of 
a correlation matrix for grouping: first attempt in analyz- 
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One useful procedure is to ascertain, first, the average correlation in each 
column of the correlational matrix Ru, where the given coefficients have been 
corrected for uniqueness. (An alternative is to count in each column the 
number of coefficients whose absolute values exceed some specified values.) 
Select the trait T, with highest mean coefficient. List all the traits whose 
correlations with T- exceed the specified value and complete the correla- 
tion matrix for the traits so selected. Eliminate from the table the trait 
which has the largest number of small intercorrelations. Repeat the elimi- 
nating process until all the traits that remain in the table have appreciable 
intercorrelations. These traits constitute a constellation. Select the trait 
whose mean coefficient is next highest and which is not listed in the group 
just formed, and proceed with it in the same manner as with T, until the 
majority of the traits have been assigned. These groupings are flexible, and 
they may be arranged to overlap. The arrangement of the traits in constel- 
lations should be regarded merely as a rough method of inspectional anal- 
ysis. The procedure here outlined can, of course, be modified in many ways. 
A note about static and dynamic factors 

Investigators who are using the factorial methods as an aid in testing 
psychological hypotheses find themselves involved not only in legitimate 
controversies about techniques and about interpretation with rival hypothe- 
ses but also in some forms of criticism that are not much more than verbal- 
isms. One of these that recurs too frequently is the charge that differenti- 
able functions or parameters that can be identified factorially are invalid be- 
cause they are supposed to be static, mechanical, atomistic, and isolated 
and that any such factors imply behavioral anarchism. In contrast, we are 
told that factors, instead of being statie, must be functional and dynamic 
“relationships” and that these must somehow embrace the whole personal- 
ity. It is doubtful whether any real issue exists to justify such talk. It is 
doubtful whether any psychologist who has good training in his subject 
»ver really looks for a factor that is truly isolated from the rest of man’s 
iw en Pai if a factor is a relation of some kind, then surely there must 
ae distinguishable things that are related; and if these things are not 
to be identified, then the relation will be obscure indeed. Я 

box problem one of the factors was the height. That was à param- 

stola ‘ed into the measurements of each box, but it does not fol- 

ic] id eae was in any sense isolated from the totality of the box. 

ee e е: is ple ег would be the objection that the boxes and their 

But to duis = static, whereas in psychology we must be dynamic. In- 

E рока м ai example, we could have used flywheels. "This ex- 
stead of using boxes ? 


inspectional methods for clusters are hardly to be pre- 


А early 
tions. The | which identify not only the clusters but also groupings in 


i rrela 
ing the со ОЙЫ, 


ferred to the factorial m! 
hyperplanes. 
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ample would have shown dynamic factors, such as momentum, the radius 
of gyration, and revolutions per second of the flywheels. Such an example 
would work just as well in differentiating between factors which would then 
refer to kinetic aspects of the objects. Factor analysis would apply as well 
in one case as in the other. Pedagogically it is perhaps better to use the 
boxes because the reader can then devote himself to factorial ideas instead 
of trying also at the same time to learn some theoretical mechanics. 

Another verbalism in the interpretation of f. 
they must be treated as "relationships." I 
without giving at least some hint about the 
Relations do not float around by themselves. 
or things that are related. In some current f. 
zation is similarly described as consisting of *relationships." If factors 


are 
the relations, then it would be interesting to have some hint about the things 
that are related, 


actors is the insistence that 
æt us not talk about relations 
entities that are being related. 

They are tied to the concepts 
actorial writing, mental organi- 


CHAPTER XV 
OBLIQUE STRUCTURE 


Primary trait vectors 

The fundamental concepts of oblique structure will be here defined for r 
dimensions. Figure 1 has been drawn to show the relations of the reference 
vectors Ap, the hyperplanes Lp, and the primary vectors Tp. Figure 2 has 
been drawn to show these relations in rectangular form. 

In Figure 2 we have the factor matrix F with elements ajm, which are de- 
termined by the factoring process from the correlation matrix. The trans- 
ation matrix A with elements Anp carries the factor matrix F into the 
tor matrix V with elements vip. This is expressed in the equation 


form 
oblique fac 


(1) Fa=V. 


The test vectors j are defined in F by their projections on a set of orthog- 
onal reference axes. The same test vectors are defined in V by their 
a set of oblique reference axes Ap. In Figure 1 the oblique ref- 
are shown, but not the orthogonal reference frame. Each 
column of the transformation matrix A gives the direction cosines of one 
of the reference vectors in terms of the orthogonal frame. 

The r hyperplanes, whose normals produce asimple structure with a test 
configuration, will be called the co-ordinate hyperplanes for the test configu- 
ration. The simple structure is defined by the test configuration and the nor- 
mals A, to the co-ordinate hyperplanes L, In Figure 1 each reference vector 
he hyperplane Lp. In particular, the reference vector 
he hyperplane Lı, and similarly for the other planes. 
The reference vector defines the hyperplane, which is asubspace of (r—1) 
dimensions in the total common-factor space ofr dimensions. 

"The intersection of any set of ey) co-ordinate hyperplanes defines a 
co-ordinate axis of the structure. In Figure 1 the тареп of {һе hyper- 
planes L, and Lz defines the co-ordinate axis Тз. The total number of sets 
of (r—1) hyperplanes that can be taken is 7, and consequently their inter- 
sections define 7 co-ordinate axes. These are of scientific interest because 
they define the descriptive categories of the simple order in terms of the 


tests. 
“The test vectors that lie in the hyperplane Lı are orthogonal to the refer- 
these tests have zero projection on that reference 


е 
ence vector Аз, and henc EORR А Tea aa 3 
vector. In Figure 2 these tests are identified by zero projections vj; in the 
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projections on 
erence axes Ap 


A, is orthogonal to t 
A, is orthogonal to t 
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column 1 of the oblique factor matrix V. Each of these projections is deter- 
mined by the cross product of a row of F and the first column of A. 

The unit vector defined by a co-ordinate axis will be called 
trait vector or a primary vector. The trait which corresponds to a primary 
vector will be called a primary trait or a primary factor. The object of a fac- 
torial analysis is to discover the primary traits and to describe them in terms 


a primary 


A 
k 
$ 
3 
5] 
E 
d 


Ficure 1 


of the tests that are experimentally 
trait vector T; is the unit vector in t] 
The subscript for each primary vec 
plane which does not cont 
is the unit vector in the i 
q7 p. In three dimensions 
gin. Their intersections de 


observed. In Figure 1 it is seen that the 
he intersection of hyperplanes L, 
tor is identical with th 
ain the primary factor, ] 
ntersection of the (r—1) hyperplanes L,, where 
there are three planes L, which contain the ori- 
termine the primary vectors T. 


and La. 
at of the hyper- 
Zach primary vector Ts 
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The intersection of all the co-ordinate hyperplanes, excepting L,, defines 
a primary trait vector, which is denoted T,. Hence T, defines the linear 
subspace which is common to all the hyperplanes, excepting Lp. The trait 
vector Т, is not contained in the hyperplane L,, but it is contained in all the 
other hyperplanes. It follows that in Figure 2 the primary trait T, is absent 
in all the tests which have vanishing entries v;, in column p of V. The pri- 
mary trait T, is present in all tests that have non-vanishing entries 0j, in 


column p of V. 


те m j p 2 " 2 
$ Amp 
9» V 
Ј Л d 
V 
Pal 
Gn 
g 
7 
FIGURE 2 


trait vector 7’, is not contained in the hyperplane Lp, 
it might be inferred that it is identical with the normal to. the hyperplane 
L,. This is not necessarily the case. If the primary traits are uncorrelated 
8 the experimental population, then the vectors T, are orthogonal, and so 
are also the co-ordinate hyperplanes Lg and their потай Ap. he this case 
the reference vectors Ap are identical with the primary vectors Tp. How- 
ever, if the primary traits Tp are correlated in the experimental population, 
diesel hyperplanes іме oblique, and their normals A, are oblique. The 


t ts of vectors Ap and T, are then, in general, distinct. 

zo sets s ; €À ены ; 

M the jetrical interpretation of primary traits may be illustrated in 
зе geon EH 


Since the primary 


pret 


350 MULTIPLE-FACTOR ANALYSIS 


three dimensions. Let the entries аўы in each row of F be augmented by the 
multiplier 1/h;. The geometric representation of the augmented co-ordi- 
nates is that each test vector is extended to unit length. 

co-ordinates are therefore the direction cosines of unit. te. 
termini of the test vectors can be represented 


The augmented 
st vectors. The 
as points on the surface of a 
hypersphere. If r=3, the test configuration can be studied graphically on 
the surface of a ball. i 

Let Figure 3 represent the test configuration, and let the points represent 
the termini of the test vectors on the surface of the sphere. Simple structure 
is shown by the fact that each point lies in one of the three a 
cles. All the tests on the are 1-2c 
since all the corresponding test vec 
can be described by three factors. 


ues of great cir- 
an be described by two primary factors, 
tors are coplanar. The whole set of tests 
Hence the same primary factor is absent 


/ 


FIGURE 3 


in all the tests along 1-2. The subspace 1-2 is defined by the direction co- 
sines of the normal to the plane 1-2. Let this normal be denoted As. The 
subscript of A; refers to the primary trait Ts, that is absent in the subspace 
Ls. The vector Аз is the normal which defines the subspace Ls. 

By analogy, the vector Д. is the normal to the plane 2-3, and Ag is the 
normal to the plane /—3, If all test vectors in the plane 1-2 represent two 
primary factors and if all test vectors in the plane 7-3 represent two primary 
factors, it is clear that the vector Which is determined by the intersection of 
these two planes represents the primary factor Which the two planes have in 
common, namely, the primary factor l. In the same manner the other pri- 
mary factors, 2 and 3, are determined by intersections of planes, 

Equation (1) is shown in rectangular notation in Figure 2. The equation 


can be written in terms of its elements, 


2 SEES 
2) X ав Amp = vj. 


m 
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In Figure 2 let the rows of a matrix T with elements £,,, represent a set 
of r primary unit vectors Ту. The rows of the square matrix Т аге normal- 
ized, since Т, are unit vectors. These are the vectors Tı, T», and T; of 
Figure 1, and they are defined as intersections of the planes L,. The direc- 
tion cosines of the first primary vector, the first row of T, are tın, tis, fis. 
The direction cosines of the normal to the plane Le, the second column of A, 
аге Aio, Age, Age. Since the vector 7; lies in the plane Le, we should have 


(3) bihis + ges + bads = 0. 


In a similar manner, since the vector T, also lies in the plane L5, we have 


(4) tadis ses + sss = 0. 

A similar substitution of the direction cosines of T in the equation of plane 
Lı gives 

(5) lin + si. si = di, 


which should not vanish, since the vector T, does not lie in the plane Lı. 
Let the constant term be denoted dı. These three equations are, in fact, the 
multiplication of the first row of T and the three columns of A, with prod- 
ucts recorded in the first row of the diagonal matrix D, as shown in Figure 


2. The other rows of T can be regarded in a similar manner, so that we have 


the matrix equation 


(6) TA=D, 


where D is a diagonal matrix with diagonal elements dp. Hence we get 


(7) TJA, 

by which the numerical values of T may be determined. The inverse of A 

Жар ыы, ‘ows to give T. : : 

= es e нба Figure 2 as a continuation of the factor matrix 
1 E. eden f Pn be regarded as if it represented a perfect test jof one 

| reda s factors, each of these tests being of unit communality. After 

of the primary ies " PREIS of these tests will have unit complexity in the 

bee ER V, in that there is only one entry d, in each row of 

; matrix V, : р av 

oblique fae е xe section of V, namely, the diagonal matrix D. 

the ‘Bonrespen’ mie between the primary factors, Rpa can be found by the 

f vx EE theorem, since T is a factor matrix for the primary 
undamental f: 

abilities. We then have 

(8) Ry = ТТ, 
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or, by (7), 

(9) Hy = DADAY’, 

which can be written in the form 

(10) Rp = DA-(A7)'D г 


and this сап be expressed as 
(11) Ry = Р(А'А)-1р. 


Equation (11) can be used to find the correl 
tors directly from A and without writing T' explicitly. The matrix multipli- 
cation (A'A) is first performed, then the inverse is found, namely, (A'A)71. 
The columns and rows are adjusted by premultiplication and postmultipli- 
cation with the same diagonal matrix D to make the diagonal entries of Ri 
unity. Let the diagonal entry of (A'A) be mp. 
diagonal entry of D is 


ation between the primary fac- 


Then the corresponding 


1 


12) = —. 
( Tan 


dy 


Since the derived pure tests T of the primary factors h 

ity, the self-correlations must be unity. 
Each diagonal element d; is the scalar 

and the corresponding reference vector Ap. Since both of these vectors 


unit vectors, the diagonal element d, is the cosine of the 
of the primary vector Т. 
then have 


ave unit communal- 


product of the primary vector T 
are 
angular separation 
› and the corresponding reference vector A,. We 


(12a) d; = ы, where q = р, 


m 
which becomes unity in the orthogonal ¢ 
In case we have an orthogonal simple structure, the reference frame of V 


is also orthogonal. Then the transformation A in (1) is an orthogonal trans- 


formation, so that A-1— A, Then in (11) A'A = А-А =], Then, by (11), 
we have „= I, as would be expected, 


ase where 7’, and A, are identical. 


Test vectors as linear combinations of primary vectors 

So far we have considered the test vee 
frame (aj, in Ё) and the primary 
in the primary factor matrix T 
test vectors which lie in the co« 


tors in terms of the fixed orthogonal 
vectors in terms of the same fr. 
› as shown in Pigure 2 
ordinate planes of J^ 


ате, tom 
- Consider again the 
igure 3, and the primary 
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factors 1, 2, and 3 at the corners of the configuration. The test vectors in 
the plane 1-2, for example, сап be expressed as linear combinations of the 
primary vectors T, and Т». We should then have 


(13) DA to = аһ, 


— 
Р 


in which A;p is the weight given to the primary vector p in the description 
of test vector j. This equation can be written in the matrix form 


(14) AT=F, 


and hence the desired weights in A are 


(15) А = FT“; 


but by (7) we have 


(16) А = FDA)", 


which becomes 
(17) A=FAD". 


By (1) we have 


li 


А =Й, 


(18) 
from which we infer that the weights by which the test vectors j can be ex- 
ar combinations of the primary factors T are proportional by 
ries in the oblique factor matrix V. When the trans- 
known by rotational analysis, the proportionality con- 
als of the diagonal entries in equation (12). 


pressed as line 
columns to the ent 
formation A becomes 
stants of D are the reciproc 
erms of the primary vectors 

an be expressed as linear combinations of the pri- 
ations гу сап also be expressed in terms of the 
actor theorem for orthogonal factors we 


The test correlations in t 
Since the test vectors ¢ 

mary vectors, the test correl 

primaries. By the fundamental f 

have 

(19) Ry = FF’, 


and by (14) we get 


(20) Ry = AT(ATY 
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(21) Rir = ATT'A', 
which, by (8), becomes 


(22) Rir = ARA! 


, 


where Rp is the matrix of corre] 


ations between the primary factors. Equa- 
tion (22) is Tucker's gener: 


alization of the fundamental factor theorem to 
the general oblique case.* The mat rix A is proportional by columns to the 


oblique factor matrix V, as in equation (18). If the primary factors are un- 
correlated, Rp, becomes the identity matrix, so that 


(23) Ry. = AA’; 
or, by (15), 
(24) fa = FTT. 
But in this case T is 


an orthogonal matrix, so that T-1— T". 
tion reduces to 


Hence the rela- 
(25) Ra = РЕ, 


which is the fundamental factor theorem for the c. 


The equation of an oblique simple structure 
A simple structure is 


ase of orthogonal factors. 


im \ а set of r oblique hyperplanes 
the origin. This set of r hyperplanes may be regarded 
whose apex is at the origin and whose s 


The equation of the hyperplane Ly is 


, all of which contain 
as a degenerate cone 
urface consists of the r hyperplanes. 


т 


(26) SER. 


m=1 


The equation of a simple struc 
ting the product of r polynom 
zero. Then we have 


(27) [x] PS мб | 


ture in r dimensions may be written by set- 
ials, like (26) for P= S 35 ‚ ^, equal to 


m=) m=1 


Y 2d = 0. 


| m=1 


* Ledyard В. Tucker, “The Role of Correlated Factors in Factor Analysis,” Psycho- 
metrika, V, No. 2 (June, 1940), 142, eq. (7). : 
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or 


This equation* may be written in the more condensed form 
r 
(28) П Pos =0. 
p=1 | m=1 


Fitting (28) to a given test configuration in which a simple structure is 


assumed, we have, for each test j, 


(29) 0305... 0. = O, 

or 

(30) II =0: 
р=1 


If the point j is in at least one of the r hyperplanes of (30), at least one of 
the r factors 0j, vanishes, and hence equation (30) is satisfied. Equation 
(28) or its equivalent, (30), is therefore satisfied by all points in the r co- 
ordinate hyperplanes of a simple structure. 

In order to determine the best-fitting degenerate cone for a given set of 


n points in a space of 7 dimensions, equation (28) may be written in the form 


of an observation equation, namely, 


(31) I| Som = pi, 
p=1 | m=1 


is the discrepancy for the point j. The best-fitting simple structure 
st Б > А 


bus which 


may be defined as that in 
Xa 
j=1 


iteri га best-fitti imple structure is the 
i кф d criterion for a best-fitting simp: 
is minimized. Hence the 


minimizing of 


ca 2; II Р D > II = ф. 


The f ion ф is then à criterion for the isolation of a simple structure. 
ıe function $ 18 6 


uggeste 
the University of Chicago. 


8 d by Professor Raymond W. Barnard, of the Depart- 
* This equation was y 
ment of Mathematics at 
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Equation (32) is an example of analytical criteria for the isolation of sim- 
ple structure. A number of attempts have been 
computing methods with analytical criteria by 
solution would be obtained with fewer gr 
methods of chapters x and тї. Lately, 
seem to be computationally feasible 
graphical work. 


made to devise practical 
which the simple-structure 
aphs than are required by the 
several solutions have been found that 
and which may replace a part of the 


The population matrix 


The population matrix Pmi for the common factors is of order rXN with 
elements tmi. Each element is the standard score Ты Of individual 7 in the 
arbitrary orthogonal factor m. In matrix form we have, for the test seore 


Sii 
(33) Sji = Хама, 
ог 
(34) = рр. 
By (14) we have 
(35) 8 = АТР, 


The test scores зе 


an also be expressed 
on the corre] 


ated primary factors. Then 


(36) ee у 
— 
т 


as linear combinations of scores 


Айг, 


r ra Д, ap, ое (о э u^ e 4 aa J 
where A jp are coefficients and Yvi Are standard scores of individuals 7 on the 
correlated primary factors D. In matrix form * 
(37) S=AP,, 
where ть; are the elements of p. By (35) and (37) we have 


(38) ATP = АР,, 
so that 
(39) DP =p, 


where P, is the population matrix foy the correlated primary factors p. 
* Tucker, op. cit., p. 141. We are using notation 4 and P for 


: he u ique cas re- 
sponding to Tucker's generalized notation Р and P. а QUA QD 
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The correlations between the primary factors can be expressed in terms 
of oblique population matrix Pe. Since ты; are standard scores, we have 


1 
(40) ты = TO tita 


or, in matrix form, 


(41) Rm = PP. 


This equation can be related to the population matrix for the arbitrary 
orthogonal frame m. By (39) we have 


(42) Ris = i (TP)(TP)' 


or 


1 эт!” 
(43) Rye = ү ТРРТ', 


which can be written in the form 


1 ppr\ g 
(44) Кы = r(s Pi )! Р 


і «n P are orthogonal by definition, we have 
But, since the factors in P are g 


Y spese ptos" 
(45) N PI Ymar 


and hence 
Pd 
(46) Rpa ; 


as previously demonstrated. 


The oblique observatio 

The relations of oblique 8 ja 
ence to the fixed and arbit rary ss ^s 
servation equation оп which multiple 


п equation і f 
tructure have been described here with refer- 


gonal frame m. The fundamental ob- 
-factor theory has been developed is 


NON E 
sji = №, Отт. 


(33) ^ 
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This is the familiar equation that we write in matrix form as 
(34) S=Fp, 


Here the arbitrary factors of P are those which appear in the f 
correlation matrix, and they are orthogonal. 

An observation equation can also be written in which the scores Sji are 
expressed directly in terms of standard scores on the correlated primary 
factors. This has been done by Tucker in the form 


actoring of a 


(36) xu Mary › 
р 


where zp: is the standard score of individual i in the primary factor p and 
where a5, are the test coefficients. In matrix form we have 


(37) 8 = АР,. 


The relations of oblique structure сап be easily derived from (36) as an ob- 
Servation equation f 


or oblique or correlated factors, 
The correlation between tests j and k is 


47 1 
(47) ae Узды, 


which has been written in the form 


e 
(48) Ry = NS. 


In the same way, 
be written 


4 T 
(40) "pq = ND twit: Ў 


and in matrix form this becomes 


the correlation between the primary factors p and q can 


(41) disces ; Pp! 


Substituting (37) in (48), we have 


(49) Ru = A (АРД(АР)/ 
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or 
1 r\ 47 
(50) Res A( PP) 
By (41) we have 
(22) Rye = АЕА’, 
which is the generalized factor theorem. This becomes 
(25) Ва = AA’ = FF’ 


when the factors of A are orthogonal. 
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А CHAPTER XVI 
FACTORIAL INVARIANCE 

Nature of the problem 
The problem of factorial invariance should be anal 
central purpose of factor analysis, the object of which is to discover a set of 
significant and meaningful parameters for describing a domain. In the box 
problem that we have found useful for illustrative purposes, the domain is 
the whole range of measurements which can be taken on rectangular boxes 
and which are functions merely of the shapes of the boxes. We have found 
that one simplifying set of parameters for this domain is the lengths of three 
adjacent edges, denoted т, у, and z, for the height, the length, and the 
width, respectively. Now let one of the measurements 5j be determined in 
large part by the height, z, and let it be assumed that in a random collection 
of boxes, selected for analysis, there is marked variance in this parameter, 
the height, Tt would then be expected that the me: 


asurement s; would have 
а certain part of its variance, a?,, attributable to the height factor in this 
collection of boxes. This is simply the Square of the factor loading of the 
factor z in test J. If the boxes Were to be arranged in order from lowest to 
highest in accordance with the factor t, we should expect to see an array of 
boxes from low to tall. Inspection of such 
the meaning of the factor x, Ordinarily, 
amining all the Measures which h i 
marked negative satur. 


yzed with regard to the 


an array would aid in guessing 
one proceeds more simply by ex- 
ave high positive, small or negligible, and 
ations, in order to guess from the 
urements what is implied by each f 

As regards the particular measurement ву, which has an appreciable part 
of its variance attributable to the height factor, its factorial composition is 
evidently dependent on the population of boxes, so that the faetor loadings 
would be affected by selection, Several collections, such as packing boxes, 
pencil boxes, Jewelry boxes, would probably vary in relative dispersions of 
the three parameters, One could imagine a collection of boxes selected so 
that they were all of the same height. The Variance in the height factor 
would then be zero, The factor loadings а; would vanish, and, in fact, the 
rank would be 2 instead of 3, Which 18 expected for most box populations. 
Hence the factor loadings cannot be expected to be invariant from one popula- 
tion to a different population. Any criterion of invari 
assumes that it is applied to analyses on the 
lent populations, * In psychological 


nature of the meas- 
actor, 


ance in factor analysis 
same population or to equiva- 
analysis this principle means that fac- 


* First ed., p. 55. 


360 


FACTORIAL INVARIANCE 361 


torial composition cannot be expected to be invariant for different a 

groups, for example, or different groups of subjects, selected by criteria ü e 
are related to the factors involved. (We shall see later that the pan sexe 
tion may remain invariant for different populations, but we are Е 
cerned with the numerical invariance of the factor loadings.) Limiting our- 
selves here to analyses that are made on the same population, we ани ех- 
pect to find that if several samples are drawn from the same population and 
if independent factor analyses are made with the same battery of bass for 
the several samples, then the factor loadings should remain invariant for the 
different samples within sampling errors if the simple structure is complete 


and overdetermined. 

Let the measurement s; be made on each one of a group of boxes, as well 
as twenty other measurements of the same domain, relating to the rec- 
tangular box shapes. Now if the measurement s; is heavily saturated with 
the height factor, we should certainly be disappointed with our factorial 
methods if the saturation of s; on the height factor should turn out to be de- 
pendent on the other measurements that are taken on the same boxes. In 
the psychological field there is a comparable situation in studying the satu- 
ration of an opposites test, for example, on the verbal factor. If that test 
calls for a verbal factor, then the saturation of the factor in the test should 
not change if additional tests are given to the same subjects and then in- 
corporated into the test battery for analysis. When the multiple-factor 
methods were being developed, these considerations led to certain demands 
as to what should constitute an acceptable factorial method. Tt is a funda- 
mental criterion of a valid method of isolating primary abilities that the weights 
of the primary abilities for a test must remain invariant when it is moved from 

The same principle can be stated as a 


one test battery to another test battery. 
fundamental requirement of a successful factorial method, that the factorial 


description of a test must remain invariant when the test is moved from one bat- 
tery to another which involves the same common factors. If a test with satura- 
tion on a factor were moved to another test battery in which that factor is 
absent, the factor would become а specific factor of the test in the second 
battery, and it would not be found by the factorial methods, which are lim- 
ited to the common factors. This limitation was made in order to make 
the factorial problem determinate. This requirement is a criterion for the 
acceptance or rejection of factorial methods. 

When a correlation matrix is factored, the result is a factor matrix with 
а set of orthogonal axes whose location in the test configuration is, in gen- 
eral, arbitrary and entirely dependent on the method of factoring. The 
s clearly dependent on the grouping of tests in the battery 
If there is a large number of verbal tests in the bat- 
and space tests, then it is evident that the 


first centroid axis 
which is being analyzed. 
tery and only a few number tests 


* [bid., p. 55. j Ibid., p. 120. 
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first centroid axis will lie close to the verbal tests. By changing the propor- 
tion of tests of different kinds, the first centroid axis can be moved about in 
the configuration. It is clear that the projection of a test vector on the first 
centroid axis will not be invariant when the test is moved from one battery 
to another battery. It will depend on the nature of the other tests with 
which it is combined in each battery. Hence the centroid axis should not 
be used for the interpretation of the factorial composition of any test. The 
same is true for all the centroid axes, and the same is also true for the prin- 
cipal axes whose location in the configuration is dependent on the concen- 
tration of tests of different kinds. Because this principle should be geometri- 
cally self-evident, it will not be demonstrated by numerica! examples, If a 


Ficure 1 


numerieal demonstration w 
loadings would be obtaine 
of different kinds of tests i 

It was for the soluti 


ere to be made, 
d by having mar 
n the two batteri 


the maximum variation in factor 
ked variation in the proportions 
es to be compared. 

m that the simple-struc- 
igure 1 we have represented two bat- 
mon factors, A, В, and C. In the first 


we have indicated with an х the approximate loca- 


axis or the major principal axis. It will be seen that 
test j would have a low projecti j 


eig " principal axis in the first 
battery, while it would h the major principal factor 
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in the second battery. Test j is far from the major principal axis in the 
first battery but close to the major principal axis in the second battery. 
If these two test batteries were factored by any appropriate method and 
if a complete and determinate simple structure were found, as shown in the 
spherical triangles, then we should find in the first battery that test j had 
a high saturation on the factor A, a small saturation on the factor B, and 
zero saturation on the factor C. The same result would be found in the sec- 
ond test battery, in spite of the fact that the two batteries differ markedly 
in the relative number of tests of each kind. The factorial composition of a 
lest in a set of primary factors that have been found in a complete and over- 
determined simple structure remains invariant when the test is moved to another 
battery involving the same common factors and in which there are enough tests 
to make the simple structure complete and overdetermined. This is one of the 
most important principles in multiple-factor analysis. It could happen that 
when a test is moved from one battery to another battery, one of the pri- 
mary factors would be represented by only two tests. Theoretically, this 
would be а common factor, but the location of this doublet factor might be 
indeterminate in the configuration of the new battery. For this reason we 
state the principle with the specification that the simple structure should 
be complete and determinate in both of the test batteries before we can have 
assurance about invariance of factorial composition. If the simple structure 
is incomplete in one of the batteries, then the locations of the primary axes 
are, to that extent, uncertain, and then the factorial composition of the 


tests would also be uncertain. 


Types of factorial invariance 
The simplest form of factorial invariance concerns the metric invariance 
of the factor matrix solution. This is the problem of ascertaining the con- 
ditions under which the numerical values of the factor matrix solution are 
е there is infinite variation in the possible locations of the ref- 
erence frame in the configuration, one should not expect any numerical in- 
variance in the projections of the test vectors on the reference axes (the 
factor matrix), unless one imposes some restriction on the location of the 
reference frame. It has been shown that the initial reference frame and, con- 
sequently, the initial numer? cal values in the factor matrix are determined 
by the method of factoring which the author is free to choose to suit his con- 
venience. If any particular method of factoring is specified in sufficient de- 
tail, then two authors would generally get the same numerical values in the 
factor matrix, but that sort of computing agreement is not of theoretical or 
scientific significance. If we impose the restrictions of simple structure on 
the reference frame and if such a frame can be found in any given test bat- 
ect invariance of the numerical values in each row of 


ag ср 
tery, then one can ex} | 
{һе iot ated factor matrix, even when a test is moved from one battery to an- 
‚һе rota а a B y 


invariant. Sine 
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other which also has a determinate simple structure in the same common 
factors. Other restrictions on the reference frame which give invariance of 
factorial composition of a test when it is moved from one battery to another 
can probably be found; but, so far, the simple-structure principle is the only 
restriction on the reference frame which has given this type of metric in- 
variance. 

A different type of factorial inv 
tion of subjec 


ariance appears in relation to the selec- 
s to whom a test battery is given, 
variance. The influence of selection on factorial 
again with the boxes. The first diagram of Figure 2 may represent the result 
of a factorial analysis of a set of measurements on a collection of boxes. The 
spherical triangle XYZ represents the 


This is configurational in- 
results can be illustrated 


structure. The cosine of the angle 


FIGURE 2 


between the primary vectors Y and Y 
mary factors in the particular collection of boxes measured. A similar inter- 
pretation applies to the other two sides of the 
pose that the same set of me: 


asurements w 
boxes. Pactorial analysis of the intere 


is the correlation between the pri- 


spherical triangle. Now sup- 
as made on another collection of 


orrelations of measurements on the 
new box collection might give such à structure as is shown in the second 
diagram of Figure 2. Here ation between the two primary factors 
X and F is much higher, as shown by the smaller angular separation between 
the two primary vectors in the second diagram. Both of the configurations 
are triangular because three primary factors are involved in both popula- 
tions. The primary factors may be interpreted in the same way for the two 
populations, but their intercorrelations are different. This should not inter- 
fere seriously with the identification of each primary factor, which can be 
done independently for cach analysis when the structure is overdetermined. 
It must be remembered that the identity ofa factor, the box height, for ex- 
ample, is not affected by the fact that it correlates differently with other 
factors in different samples of boxes that may be chosen for measurement. 


the corre] 
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Confusion about this rather simple idea has been responsible for misinterpre- 
tation of factorial results. 

In each of the two box collections the measurement j was in both test 
batteries. In the first diagram of Figure 2 one infers from the location of the 
measurement j that it is a function of both X and Y and that it does not in- 
volve the parameter Z. Furthermore, the fact that the test vector j is nearer 
Y than X makes it evident that the parameter Y has greater influence on the 
variance of j than does the parameter X. The score function s; might be 
some function like (2y4-2), which would lie closer to Y than to X because a 
greater part of the total variance of this score function would be attributable 
to Y than to X. But notice also that the relative location of the measure j 
is the same in the second diagram of Figure 2. This result is caused by the 
fact that, no matter what the correlation between the parameters X and Y 
happens to be in the particular population being analyzed, the score j is a 
function of X and Y but not of the parameter Z. Hence it must lie in the 
side XY of the spherical triangle. Furthermore, no matter what the correla- 
tion rzy happens to be, the score j is closer to Y than to X because of the na- 
ture of the score function s;. Here we have an example of configurational 
invariance. The factor loadings might change markedly from one popula- 
tion to another; but, if the.same test battery is used on both populations, 
the configuration should be invariant. As a matter of fact, this is a far more 
important consideration in factor analysis than the invariance of the factor 
loadings. If the object is to identify a set of significant parameters for the 
description and understanding of a domain, it is more important to be able 
to make the identification as to the nature of the parameters than to ascer- 
that, in this particular collection of individuals that we happen to be 
studying, the correlations between the parameters have certain numerical 
values and that the saturations of some measure j are such and such for this 
particular group of individuals. Later, when the nature of each primary 
factor becomes well understood, it may be a matter of great importance to 
ascertain the numerical values for particular groups of subjects in the form 
i like, but not at the exploratory stage of investi- 


f practical norms and the c 
район. factorial methods have been devised. At this stage we 


et of concepts or parameters in terms of which the do- 
od. The numerical values should then be regarded as 
being of three kinds, namely, those that are significantly positive) сел Шы 
vanish or nearly vanish, and those that are significantly negatis A ; печ 
way of stating the same idea is to point out that it is the factor pa “ooo 
is of importance rather than the factor matrix with ita ao " n 
Configurational invariance ean be represented algebraica y ^ eras Е 
the test vectors as linear combinations of the primary зеге р 2 felt 
tions will be described in chapter xix, with numerical examples of the re- 
d multivariate selection in factor analysis. 


tain 


gation for which 
are groping for some 8 
main may be understo 


sults of univariate an 
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Suppose, now, that a third collection of boxes is chosen for analysis, with 
the same set of measurements, and that all these boxes have the same height P 
x. The configuration would then collapse to the form shown in the third 
diagram of Figure 2. The factor X disappears because it contributes nothing 
to the variances of the measurements, and the rank of the system is 2 in- 
stead of 3. The test vector j is then identical with the parameter Y because 
it is a function of Y. The contribution of X is merely a constant in the score 
function with no contribution to the variance. The factor X can hardly be 
expected to be identified when it is completely absent from the variances of 
the scores that are being analyzed. Of course 


; it does not follow that the 


factor X is absent from the individual members of the population. It is 


equally present in all of them and therefore does not contribute to their in- 
dividual differences. 

Another case of factorial invariance is that in which two different test 
batteries are given to two populations. If the simple structures are complete 
in both batteries in the same common factors, factorial analyses should give 
the same configurations with different arrangements of the test vectors, un- 
less some of these are common to the two batteries. The identification of 
the primary factors would be made independently for the two 
it should generally be possible to identify the same primary 
two batteries, even if they have no tests in common, This type of result is 
the most convincing form of demonstration of the existence of primary men- 
tal abilities and their interpretation, especially if the factorial compositions 
of new tests are predicted at the time of their construction and later verified 
with factorial results, Tt is best in such studies to include several tests from 
previous studies as linkage, so as to make the demonstration of the 
and nature of the primary factors more convincing. It could h 
such studies that the new battery would revea 
ously known. The result might. be 
factors or the breakdown of 
tors. These r 


analyses, and 
factors in the 


identity 
appen in 
ul common factors not previ- 
a reinterpre 
a primary factor in 
esults are dependent on the 
are unlikely to be completely de 
exploratory studie 


One form of factorial invariance that has been legitim 
that in which a small test battery is sele 


tation of the older primary 
to several new primary fac- 
selection of test batteries, which 
terminate for all the factors, especially in 


ately demanded is 

eted from a larger one that has been 
* Some time ago the writer was disappointed to find that a certain new test, called 

“Designs,” did not show large saturation on i actor which was the 

object of a factorial analysis. Li 

a new perceptual factor concerr 

included in a study with other tests, sue 


If that is the ease, then 
actor сап be made with a little 


arge amount of experiment 
al faculties. 


more confi- 
al work, we shall 
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shown to have a complete simple structure. The analysis of the smaller bat- 
tery could be made from a section of the larger correlation matrix. In such 
studies it should be expected that the smaller battery would show factorial 
invariance of the factor loadings if the smaller battery were selected so as to 
contain its own complete simple structure. If one of the hyperplanes is un- 
stable in either the larger battery or the smaller one, or both, then factorial 
invariance cannot be expected, because there would then be no criterion for 
locating the reference frame which determines the numerical entries in the 
factor matrix. 


Rank order and the normalized scale 

In most psychological tests the raw score is merely a numerical index of 
‘unk order. The reason for this interpretation of raw scores in psychological 
and educational tests is that there is no rational origin for the scale, and 
there is no defined unit of measurement which can in any fundamental sense 
be regarded as representing equal increments at the different parts of the 
scale. An exception is occasionally found in which all tests in a battery are 
scored in terms of time or of some natural spatial unit in which a rational 
origin can be used. The raw score interval from 20 to 30 points is not regard- 
ed as representing in any sense the same increment of ability or achievement 
as the interval from 80 to 90. The origin of the scale is quite arbitrary, being 
determined by the number of easy items, by the scoring formula which is 
more or less arbitrary, and by such arbitrary matters as the time limit. The 
increments at the different parts of a distribution can be modified to repre- 
sent large or small increments in ability by the distribution of difficulty of 
the items selected for the test. The shape of a distribution of raw scores can 
be made normal or positively or negatively skewed, at will, by merely al- 
nits and the distribution of difficulty of the individual test 
umstances there is little justification for dealing with 
e than rank orders for the subjects in an ex- 


tering the time lin 
items. Under these eire 
the raw scores as anything mor 


perimental group. : . o : | 
The problem is sometimes raised as to whether the distribution of intelli- 


gence in the general population 18 normal, but such a question has little 
meaning unless it is modified. One can ask, for example, whether a scale ex- 
ists such that the distributions of performances of successive age groups are 
normal on the same scale. That is the problem of absolute scaling, and it can 
be rather easily solved.* In dealing with the distribution of raw scores for a 
psychological test, one has the privilege of defining the scale so that a par- 
ticular distribution of raw scores is normal, or one can define it so that the 
distribution takes any form one chooses. That is a question of definition, 
It is not a question of fact or a question to be determined experimentally. 


+1. I. Thurstone, «А Method of Sealing Psychological and Educational Tests," 
Vol. XVI, No. 7 (October, 1925). 


Journal of Educational Psychology, 
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For example, an author has the privilege of defining his measure of stature 
as the logarithm of height in inches if he so chooses. The rank orders of his 
subjects remain the same. Similarly, one might decide to use the quartile 
range instead of the standard deviation as a measure of dispersion, or the co- 
efficient of alienation instead of the correlation coefficient. These are exam- 
ples of monotonic functions. 

If a transformation is made by which the distribution of raw scores be- 
comes normal, then one has, in effect, merely altered the numerical values 
of the individual scores so as to retain the same rank order of the subjects. 
Such a scale is a monotonic function of the original raw scores. For any given 
frequency distribution of raw scores one could determine, by empirical 
curve-fitting, the monotonic function which transforms the given raw 
scores to a new set of scores for which the distribution is normal. This would 
be an awkward way to solve the problem. A simpler way is merely to tabu- 
late the percentile ranks of the given raw scores and to assign to each raw 
score the standard score which corresponds to the percentile rank. This is 
determined directly from a probability table, such as the Kelley-Wood 
tables. The resulting scores are called normalized standard scores or simply 

normalized scores, They should not be confused with standard scores deter- 
mined by a linear transformation, which represents merely a translation of 
the origin to the mean and a unit of measurement equal to the standard de- 
viation of the distribution. This distribution of standard Scores retains the 
shape of the original distribution of raw scores, whereas normalized scores 
give a normal distribution, no matter what the shape of the original dis- 
tribution may be. In both cases the rank order of the subjects is retained, 


It would be possible to carry out factor analysis with r 


ank orders instead 
of scores, 


either the raw scores or the normalized scores. Such a procedure 
would probably be discouraged because, if the underlying distribution is 
assumed to be unimodal of any shape, then the rank orders in the middle 
range should represent smaller increments than the rank orders at the ex- 
tremes of the distribution. Hence the procedure of using normalized scores 
in factoring is probably more appropriate than to use either г 
the arbitrary raw scores. These considerations ma 
to normalize the raw scores before the fae 

The procedure of normalizing the raw scores before factoring introduces 
a form of invariance. All monotonic increasing functions of the raw scores 
retain the same rank order of the subjects. Hence all monotonic increasing 
Junctions of the raw scores are factorially identical when the scores are normal- 
ized before factoring. Empirical comparisons have been made between sev- 
eral methods of treating the original scores. These can be listed as follows: 


directly from the raw scores 
alized scores 


ank orders or 
uke it a desirable practice 
tor analysis is made. 


1) Factoring the correlations obtained 
2) Factoring the correlations of norm 
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3) Factoring the correlations of new scores obtained with a separate ar- 
bitrary monotonic transformation for each test and without normal- 
izing 

4) Factoring the covariances of the raw scores 

Comparisons of these different ways of treating the scores in factoring have 
shown that if the factor pattern is that of a complete simple structure, the 
different treatments of the scores in the factoring give essentially the same 
result. In general, the structure is retained in clearest form by the procedure 
of normalizing the raw scores before factoring, and that is the recommended 
procedure. The different treatments of the scores give different correlations 
or covariance entries in the correlation matrix, but the rotated factor ma- 
trix and the configuration determine whether the factorial results are the 


same. 


Invariance of factor loadings 
A numerical example of the invariance of factor loadings will be described 


A later chapter will be devoted to the effects of selection on factorial 
configurational invariance. In order to make this numerical 
example as concrete and direct as possible, it was designed asa three-dimen- 
sional box problem. Measurements of a random collection of thirty boxes 
were actually made in the Psychometric Laboratory and recorded for this 
numerical example. The three dimensions, т, y, and z, were recorded for 
each box. A list of 26 arbitrary score functions was then prepared. This is 
ast column of Table 2, which shows, for example, that test 4 
consisted of the area zy of one side of the box. | 

The data were analyzed in several ways, but only the analysis for the nor- 
malized scores will be presented here. After the score functions had been 
computed for each of the thirty box shapes, the distributions were normal- 
ized so that normalized standard scores were used for the computation of 
orrelati efficients. The resulting correlations are shown in Table 1. It 
а r he table of correlations that several pairs of columns are 
will be me p m This is the result of normalizing. For example, 
oe — Б in the correlation matrix are identical. Test 15 was de- 
е й s = ү 16 was defined as 2/т. The rank orders for these two 
айша, poe е 1 except for direction, and consequently the correla- 
formulae " bu annt same way the score functions z and 
tions are identical exe 


t for sign. In the s 
11 been identical in the correlation if 2? had been included in the 
x? would have bee а 
һайегу. 


‘The correl «matrix is shown in Table 2. The last column of the factor 
the resulting factor “o mmunalities. A frequency distribution of the third- 
matrix shows the ¢ Table 3, and it is seen that the average of the 


als is shown 1 


here. 
results and to 


shown in the 1 


à sowas factored by the group centroid method, and 
ation matrix was 


factor residu 


58 8° 88" 25° 28g 

SZ: L6 89° $0 

eg" > 5b 
90^ 


THD JY u01)0]2440, ) 


г 190.1 : 
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absolute values of the residuals was .0116, which hardly justifies extracting 
another factor. The fact that the three factors were sufficient to account for 
the intercorrelations agrees with the fact that three parameters, т, у, and z, 
were used in writing the 26 score functions of this battery. 


Table 2 
Group Centroid Factor Matrix Factor Matrix after Rotation 
I II III ie x Y 2 Variable 
E 33 .98 oL ol z 
37 .97 92 .01 y 
64 .98 .05 :91 2 
48  .99 .64 —.03 ту 
24 -99 00 .62 zz 
20 1.00 60 .58 уг 
43 1.00 38 — 01 xy 
43 197 79 01 xy? 
03 .99 —.01 41 zu 
44  .98 —.02 .79 22° 
01 .98 74 .40 yu 
.98  .98 41 .74 yz 
09 .97 —.77 06 a/y 
.09 .97 77 —.06 y/z 
.80 .99 02 —.73 2/2 
.80 .99 —.02 .73 z/z 
85 .96 80 —.76 y/z 
.85  .96 —.80 ‚76 2/1 
.48 .97 70 —.03 2+2, 
32 1.00 —.04 69 224-22 
.19 1.00 60 58 Qy +22 
AT 94 69 —.03 Va +? 
32 .96 —.001 .68 Vez 
16 .97 pr .55 ye 
.01 .98 3 .45 туг 
.02 93 .51 .46 Vett? 
Table 3 
Third-Factor 
Residuals 
سے‎ 
p / 
те ME یا‎ Е 
.05 1 
104 4 1" 
;03 9 210: 0116 
e 37 N -0116 
.001 7 
.00 84 
201 70 
—.02 30 
oe dA 
—.04 2 
—.05 
—.06 1 
pou 
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The method of extended vectors was applied to the factor matrix of T'a- 
ble 2, and the resulting diagram is shown in Figure 3. The three planes are 
here immediately evident. All the points lie on or close to the sides of the 
triangle, except 25 and 26. Referring to the list of score functions in Table 2, 
we see that these two test measures were the volume and the main diagonal 
of the box, respectively. Both of these mes 
the parameters, and consequently the tw 
the configuration. Hence they do not со 
ordinate planes of the simple structure, 


isures are functions of all three of 
© points 25 and 26 should lie inside 
ntribute to the location of the co- 
The list of score functions was so 


2 Plane 


FIGURE 3 
designed that most of the scores were functions of one 
and only two of the Scores were functions of 
Hence a simple structure Was incorporated in 
resented by the fact that all but two of the scor 


figuration of Figure 8. By the method of extended vectors the transforma- 
tion A was determined, and the factor matrix after rotation is shown in 
Table 2. Diagrams for the two-dimensional Sections are shown in Figure 4. 

In Table 4 we have the correlations of the three primary factors T, Y, 
and z, as determined from the battery of 26 measures. In the same table we 
have the correlations of the orig; 5 à 


hs inal dimensions of the t 
Che two sets of correlations agree closely, 


or two parameters 
all three of the parameters. 
the test battery. This is rep- 
es defined the triangular con- 


hirty box shapes. 
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When this analysis had been com S 
selected from the longer list of 26 к a p ү ji uum eed 
was made with the restriction that it should be balnei. jt Е 
m the larger battery. The larger battery was equally Ma eru 
пее primary factors, x, y, and z. For the purpose of this damio t | oe 
smaller battery was sclected so as to contain a relatively seach iq 
J эб; 


Р : 
*| le 
e ° w 
ee Pe ا‎ .* * ev 
4 ° eg ° 
е е e ә eo m 
| 
— е о 9 
о e 9, o MEI چا‎ le ° 
e. шы i 
| 
| 
Cos. 
s ° 
° Ы . ° 
2, 
e 
E А ° 
° 
M pr 
4 ы ° 
° 
° e ° ot Р 
p LJ 


FIGURE 4 


ctor y and only a few tests of the other two 
faetors. The second battery was also selected with the restriction that the 
simple structure should be complete and determinate. This selection could 
be easily made from the diagram of Figure 3. The smaller battery was then 


independently analyzed. 
The smaller battery is show: 


tests involving the primary fa 


nin Table 5. The correlations were taken from 
the larger correlation matrix of Table 1. The group centroid method of fac- 
toring was used, but any appropriate factorial method could have been 
used. The mean absolute value of the third-factor residuals was .015, which 


Table 4 


Correlations between Primary Factors 


x 1 2 
X 1.00 .22 -21 
) i22 — 1.00 .30 
2 21 -30 1.00 
X Е $ 
X 1.00 .26 .23 
Y .26 — 1:00 .30 
Й .23 .30 1.00 
Table 5 
Factor Matrix aft 
Growp Centroid Factor Matriz ab gi e 
I i ш D: x p d 
d : ————— 


4 .95 4 

б .82 -43 —.36 98 
Ys .91 —.29 .82 1.00 
8 .92 —.18 -—:96 95 
19 -94  —.32 =, 99 
22 ЭЗ? —.31 ~ 19 97 


Table 6 


Correlation between Primary F 


‘actors 


x Y 2 
Х 1.00 .18 .26 
) .18 1.00 -37 
А -26 -37 1.00 
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did not justify extraction of a fourth factor 
ee y с j ctor. The method ч 

bs pr уы Ше енн matrix of Table 5, and the кыз эск is 

о Me a А : 8 ане factor matrix after rotation is шл 

wa en iid is › г 5. The figure and the rotated factor matrix show 

Dn te cture is present in the smaller battery, as was to be ex 

E . The correlations between the three primary factors йс 
y 


FIGURE 5 


the smaller battery are shown in Table 6. The several tables can Dexeen to 
be very similar. 
The invariance 0 
of the factor matrices 
factor loadings for the over 
example should serve to illustr 
tion of a test is invarian 
battery if the simple structure is 


an be seen by direct comparison 


f the factor loadings c 
will be seen that the 


after rotation in Tables 2 and 5. It 
lapping tests are very similar. This numerical 
ate the principle that the factorial composi- 
t when it is moved from one test battery to another 
determinate in both batteries. The in- 
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variance is independent of the proportions of tests which are heavily sat- 
urated with each primary factor, so long as the structure is defined in the 
same common factors, 

A distinction must be observed in the discussion of factorial invari 
between the characteristics of factorial methods and the char. 
the data to be analyzed. In setting up a criterion for what is to constitute 
an acceptable factorial method, the chief concern is to insure that the meth- 
od is adequate to discover what is determinate in the data. Factorial meth- 
od has not been designed to guarantee the discovery of factors which are in- 
determinate in the data. If the purpose of a factorial investigation is to 
study a particular domain or a particular primary factor, then the test bat- 
tery should be designed to fit a crucial experiment for the hypotheses to be 
examined. The investigator must use his ingenuity in covering the domain 
So as to make the structure determinate. Too often, factorial studies seem 
to be made on correlation matrices which just happen to be available, with- 
out reference to the likelihood that they will produce significant results, 
Occasionally, such studies do produce interesting findings, but it is safe to 
say that every such exploratory study will indicate the necessity for addi- 
tional studies of test batteries specially designed to test the hypotheses that 


may appear in the exploratory study regarding the nature of the underlying 
parameters or factors, 


ance 
acteristics of 


CHAPTER XVII 
ALTERNATIVE METHODS OF ROTATION 


The problem of finding an analytical criterion for the reference frame 


The methods of locating a reference frame for a simple structure that 
have been described have all been graphical, and they have all depended on 
the selection of a subgroup of test vectors for defining each hyperplane. The 
graphical solutions in n-dimensional space are generally slow, and they re- 
quire a considerable amount of labor. Ever since the simple-structure solu- 
tion was found for the problem of locating a reference frame, there have 
been many attempts to arrive at the solution without the graphical proce- 
dures. Scores of proposed solutions have been tried on a variety of factorial 
problems, but, with a few exceptions, they have not been published. None 
of them, so far, has entirely satisfied the desire to find a routine method 
which can be applied to a set of data without calling for any judgment on 
the part of the computer. In this chapter we shall describe several pro- 
cedures for locating the simple structure when it exists in the test battery. 
They all require considerably less labor than the graphical methods. They 
are not entirely automatic, in that one cannot do the computations blind- 
folded, as it were; but they are in some degree successful, in that a consider- 
work is done without plotting graphs. In fact, only a 
ed, in comparison with either the method of two- 
he method of extended vectors, which is really a 


able portion of the 
few graphs need be plott 
dimensional sections or t 
method of taking three-dimensional sections. In the final acceptance or re- 


jection of a simple-structure solution it is still the appearance of a set of 
graphs that determines the answer. 


The problem of finding an analytic 
have been made, сап be briefly stated as follows: Let it be 
as been factored so that a factor matrix of r columns is 


correlation matrix he 
available. Let it be assumed, further, that a simple structure exists in the 


test battery. It would be desirable to be able to write explicitly the direc- 
tion cosines of the transformation A in terms of the factor loadings and to 
be able to solve for this transformation directly. 
The methods that have been tried consist gener: 
tion ¢ of the factor loadings and of the direction cosines of the transforma- 
tion matrix A, which is to be maximized or minimized at the desired solu- 


each column of A. In effect, this general procedure consists in de- 
whose projections on one of the reference 


are the tests which have vanishing projec- 
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al solution, for which many attempts 
assumed that the 


ally of writing some func- 


tion for 
fining a subgroup of test vectors 
axes are to be minimized. These 
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tions in a column of the oblique factor matrix V. Since the ideal solution is 
oblique, the solution for each reference vector is independent of the solutions 
for the other reference vectors. The choice of a subgroup for defining each 
hyperplane is usually done graphically, but this work сап be done, at least 
in part, without graphical aid. 

In practical work there frequently arise various forms of ambiguities that 
cannot be resolved by statistical methods if the structure is incomplete. 
Since it happens not infrequently that the Structure of a test battery is un- 
stable in one or more dimensions or else indeterminate as to the exact rela- 
tion of one or more factors to the test battery, it is best for the author to 


factors that appeared unstable in the initial studies, In Some factorial prob- 
and so convincing that it leaves little room 
reference frame. It is for such test batteries 


ave a completely objective analytical procedure 
for locating the reference frame. 


A successive approximation method of rotation 


The theory of the method will be given first, after which numerical exam- 


ples will be given. In using this method one Starts with a trial reference 
vector that will be denoted A, as in Figure 1. The projection of each test 
vector j on the trial vector A, is computed. It is the scalar product ЈА = 
These projections constitute, i 


matrix V. A frequency distrib 


ja 
n a sense, a trial column of the oblique factor 


4 ution is made of the values of б. A subgroup 
of tests is selected from this distribution whose projections on a new trial 


reference vector are to be minimized. In selecting a subgroup, one may take 
all tests whose Projections are in a range of, say, +.30 or + -20, if a positive 
manifold is not to be imposed on the simple structure. If it is assumed that 
à positive simple Structure exists, then a Subgroup is selected which in- 


cludes all negative Projections, so that the range chosen would be, say, 
+.30 to — 1.00. 


The first trial vector ma 
trix. One selects a test whos 


of seventeen tests, selected from one of the 


lems, a Inspection of the columns of that correla- 
tion matrix indicates that tests 1, 2, and 3 should be suitable trial vectors. 
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We choose one of these and normalize the test vector so that it is of uni 
length. This is the first trial vector Aj, which is shown in Figure 1 реи 
Р In denoting the successive adjustments of a reference vector, we ahs П call 
its initial position Ao. After the first adjustment it will be дейда ^n He 
the second adjustment As, and so on. In the computations for an; on " 4. 
justment, the given position of a reference vector will be called ^w аай бв 


FIGURE 1 


denoted Ap, which becomes A, for the next adjustment, 


and go on until a satisfactory solution has been found. 
A correction vector C, is to be found, which is orthogonal to A,, as shown 


in Figure 1. One such correction vector is found for each of the planes de- 
termined by the trial vector A, and one of the fixed orthogonal axes M of the 
frame, defined by the given factor matrix. Hence there will be r correction 

ed in à common-factor space of r dimensions. Let 


to be determin 
r M in the figure be one of the centroid axes or one of the axes 


new position will be 


vectors Co 
the unit vecto 
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defined by factoring the correlation matrix. It is here drawn oblique to As 
because, in general, the trial vector A, is oblique to the centroid axes. The 
projection of A, on M is Ama, which is one of the 7 direction cosines of Aq. 
Figure 1 has been drawn to represent only one correction vector Co in the 
plane of A, and one of the fixed orthogonal axes M. Then 

(1) AM = А 


тт. 

In this discussion, vectors will be denoted by capit 
by lower-case letters except for the test vectors 
in other chapters with the notation j. LetJ be th 


al letters and scalars 
which have been discussed 
€ projection of a test vector 


Table 1 
Correlation Matriz 

1 2 3 4 5 6 7 8 9 10 11 12 22 23 24 25 26 
1 .06 .56 
2 173 175 
3 74 .74 
4 87 83 
б 7 80 
7 S8 .9 
8 
9 
10 
11 
12 
22 
28 
24 
25 
26 


j on the plane of A, and 
is а, Which is a cell en 
unit vector, 


M, the plane of the fig 


i ure. The projection of j on M 
try in the given factor 


matrix F. Then, since M is a 


(2) MJ = аы. 


The projection of j on Ay 
adjustment in the positio 
unit vector, 


(3) 


1S dja, one of the projections to be minimized by an 
on of the reference vector. Then, since A, is also a 


ЈА, = Bye 


Applying the orthogonal correction vector С 


‘0 to A, in the plane of A 
and A7, we have the long reference vector Aj, n : i ` 


amely, 
(4) Ag+Co=A,. 
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The direction of A; can also be defined conveniently by a correction vector, 
Cm, Which is collinear with Л/.* Let i 


(5) Ca = —sM. 
Then 
(6) : As = Ag Cn, 


where A, is collinear with Az. We can write 
(7) A, = INS, 


where A, is a unit reference vector and ¢ is a stretching factor. 
The projection of the test vector j on the unit reference vector A, is vjr. 
It is the scalar product 
(8) JA, = 0. 
correction vector Cm is to be determined so that the pro- 


The length of the с 
he selected subgroup are minimized. Substituting 


jections vj, of the tests in t 
(7) in (8), we obtain 


(9) 


UM 9j, 


and, by (6), 


(10) tJ (Ag + Cm) = 0. 
By (5) 
(11) U(M — sM) = vj, 
which becomes 
(12) tJ Aq — ЈМ = 0. 
By (2) and (3), 

(13) irj, — {8 ут = Vir. 
is to find the length s of the vector С„ so as to minimize 


The problem now 
ане quaring both sides of (13), we get 


ea s: 
the projections vjr- 5 

à 
(ij, — 2Esvjaltim + sam = vis. 


(14) 


* The vector A; is 1 
defined with r such eor 


h only one correction vector, Co. It will later be 


here defined wit 1 с 
one for each centroid axis M. 


rection vectors, 
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Summing for all tests in the selected subgroup, we have 


У 22 B osx % жй 
(15) D У 22. — 205 jq lin + Ps 2 а = c ,UrT м. 
j 7 2 j 


Writing the partial derivative of u with respect to s and setting it equal to 
Zero, we obtain 


B NUS. ay 
(16) —2P S vas, -- ЛЭЭ Gn = 0, 


jm 
j 


from which 


"m 
25405, 
< Ja jm 


(17) c= => A 
Sa. 


The values of аы are known in the given factor matrix Ё, and the values of 
Yiq are known, since they are the projections of the test vectors in the sub- 
group on the first tria] reference vector Ал. Hence the length s of the correc- 
tion vector C,, is known and also the direction of the vector А,. 

If the given projections v;, and аһ are plotted on 
tion paper with the axes A, and Л/, drawn at right 
test vectors j will be repre 
A, can be drawn so as to 1 
through the points, by the method of least squares, will be orthogonal to A, 
through the origin. The slope of the best-fitting line is 8, which is also the 
length of the vector Cm, by which the projections of the test vectors in the 
subgroup on the vector A, are minimized. 

With the r correction vectors C, known, we Proceed to add them to А, 
and thereby determine а vector to be denoted Au, which, when normalized, 
becomes the new unit reference vector Ap. The vector Az is defined by the 


rectangular eross-sec- 


angles, the termini of the 
sented by points. On such à graph the location of 


ninimize the projections Vj. The best-fitting line 


equation 

(18) AL = kA, , 

where k is a stretching factor by which КА, has Unit projection on A, By 
(5) and (6) we have 

(19) 


k(A, = sM) = A; . 


Since A; has unit projection on A, and since A, is a unit vector, we have 


(20) 


AzA, = 1 5 
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or, by (19) and (20), 


(21) k(A, — sM)A, = 1, 


(22) kA? — ksMA, = 1. 


Ву (1), and since Aj = 1, we get 


(23) k — km = 1, 
so that 

1 
(24) k- D u 


One of the direction numbers of Az is represented in Figure 1 by the pro- 
jection of Az on M. This is Nm The value of Xm can be written 


(25) АМ = wis 
By (19) and (25) we have 
(26) (А, = SM)M = №, 
ог 

(27) КА М — ksM? = №: 


and by (1) this becomes 


(28) Oma — 8) = Amt + 


Substituting (24) in (28), we get 


Ama — 8 
Agi 


(29) 1-— £5 ' 


1 numbers Amı Equation (29) is evaluated 
separately for each of the r direction numbers № of Az. The direction co- 
sines X, of the trial reference vector A, are known, and the vector length 8 
of C, is determined by (17) for each of the r columns of the factor matrix F. 

When the direction numbers Amı of Az have been computed, this vector 


which determines the directior 
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can be normalized to unit length, and it can then be used as a new trial 
reference vector A,. Its direction cosines are then given by 


(30) Lut = ==. 


It has been found that this method of locating the co-ordinate hyper- 
planes is more rapidly convergent if the vector A, is adjusted so as to be 
orthogonal to the centroid vector A. of the subgroup s. The adjustment can 
be made on the vector A, in the plane of A, and A, so that its adjusted posi- 


tion A, is orthogonal to Ae. The adjustment vector may be denoted X, so 
that 


(31) A+E=A 


иу 


and, since A, is orthogonal to Ac, we have 


(32) АА, = 0. 


By (31) we have 


(33) (А, + E)A, = 0. 


я А 
The adjustment vector E can be Written as a fraction of Ac, so that 


(34) E = bhe, 


where b is a scalar. Then 


(35) (A + РАЈА, = 0, 
or 

(36) AA. + DA? = 0, 
so that 

(37) b= — А4. 


A? * 


Then, with b known, we can write, from (31), (34), and (35), 


(38) Жы зе AR А4, А, 
Ac 


Which defines the vector A,. 


— 2 A 


| 
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Equation (38) can be written in terms of the r components of Ay. We 
then have i 
AS Nai 
т a 
(39) Ми = Amt — “Sa Ame 


where Aw, are the direction numbers of Au, and Ал, are the direction num- 
bers of the centroid vector A.. The components of A, can be determined 
from simple summation in the factor matrix Ё over the subgroup s. We 


then have 


(40) Ame = > oe Р 


s 


which may be denoted briefly Уа on the computing forms. 
3 


The scalar product A;A« can be written in summational notation as 


E È3 mm 
— 


m 


(41) МА, 


and the denominator А? can be similarly written as 


(42) к= >= x (Lam) 
If we let 
(43) ا‎ d 


we can write the computing formula as 

(44) Ми = Ant не: 

has been determined, it is normalized and then becomes 
A, If this unit reference vector is to be adjusted 
«ting-point, Ao for the next trial. 

s to the numerical work, which can be easily done 
tion in factorial theory. The computational work 


ical examples. 


When the vector Au 
the new reference vector 
again, it becomes the star 

This method is simple & 
by a clerk without instruc 
will now be illustrated by numer! 
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Numerical example No. 1. 


For the numerical example of this method of locating the co-ordinate 
hyperplanes of a simple structure we shall use a seventeen-variable box 
problem selected from the numerical example of the previous chapter. This 
battery represents a three-factor positive structure. The correlation coeffi- 
cients are shown in Table 1. A corresponding factor matrix F is shown in 
Table 2. The computations are shown here without the summational checks 
which are essential in practical work. 

In Table 3 we have the same factor matrix recorded with only the first 
digit. It is used in some of the cross products in order to simplify the com- 


Table 2 Table 3 
3 Factor Matriz F, 
Factor Matrix F (One Decimal) 

I II III I II III 

1 1 -6 —.6 3 

2 2 E d 5 3 

3 3 d 0 —.6 

4 4 8 0 4 

5 5 Ө cds шы 9 

6 6 8 4 —.2 

7 7 8 —.3 E 

8 8 .8 2 E! 

9 9 S8 —.5 .0 

10 10 28 42 d 
11 11 .8 4 .0 
12 12 .8 2 —.3 
22 22 .8 0 4 
23 23 B =.3. —28 
24 24 8 3 —. 
25 25 :9 0 .0 
26 26 .9 1 .0 


putations, since one significant, figure for the weig 
purposes of this method. 


When the factor matrices hay. 
sheet is prepared as shown in Т 
dependent test vectors as there are common f 
used as the initial trial reference vi 
justed by successive approximation until we find a set о 
that defines a simple structure if it exists in the data. 


hts is sufficient for the 


a of having a wide range of correlation 
nns of the correlation matrix and also 
an appreciable number of relatively low coefficients. These three test vec- 
1e computations. The first section 
of Table 4 shows the factor loadings of these three tests, copied from Table 2. 
These three test vectors are normalized, and the resulting direction cosines 
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are shown in the columns A4, Az, Ac of the transformation mat.ix Ap. These 
three unit vectors are the normals which define the three trial planes, A, B, 
and C. 

In the matrix Vo of Table 4 we have the projections of the normal-length 
test, vectors on these three reference axes. These are obtained by the com- 
putations FA,= Vo. In the same table we also have three frequency dis- 
tributions of these projections, one for each of the three trial vectors. The 


T'able 4 
Example 1. Initial Vectors 
Trial Test Vectors Matrix Ao С=л'А 
Test 1 Test 2 Test 3 AA Ав Ac AA Ав Ac 
I 65 .74 .75 r .66 .75  .76 Aa 1.0 .25 .25 
Il | —.67 .53 .06 II | —.68 .54 .06 An .25 1.00 .36 
IH .99 .97 —.04 III „88 .38 —.65 Ac .25  .36 1.00 
Matriz Vo Distribution Do 
د‎  —— 
A p © A 8 g 
1 99 .25 .24 9 3 3 3 
2 25 .98 .85 8 5 S 
3 24 .35 -99 ue 2 а 1 
4 76 .81 -35 6 3 2 2 
5 тт .95 .80 5 3 
6 24 .81 -83 4 EE 
7 93 .60 -34 .3 5 р a 
8 56 .93 .39 ki 
9 QT U .62 a 
10| .59 .3 2 +0 
11 23 .91 -69 = 
12 25 .06 -93 ЕЕ: ЕЕЕ 


i ; their projections in each distribu- 
st vi 's are designated by their projec i 
icd ней s. Each of the three subgroups is to be used 


lowest six t 
group 


tion as constituting à sub; 
for defining a plane. 


ix ic 7 1 f the angles be- 
М trix C, which shows the cosines 0 g 
Finally, we record the ma 


i tion of this matrix shows that 
C tua reference axes. Inspec ` 
tween the Luck mere close together to define the same plane. | 
по two of them а i Eo Table 4 have been completed, a data sheet is 
When the compu ta heton section of this table shows the computations 
f Hi n decies of the A-plane, and it will be described in some detail. 
ог the adjustment 


-osts in the subgroup for plane A. In the dis- 
ip i " r the six tests in t 
A strip is prepared for 
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tribution of Table 4 we see that the subgroup for the A-plane contains all 
projections below .30. In column A of the matrix Vy we find that these six 
tests are 2, 3, 6, 11, 12, and 24. The factor loadings of these six tests are 
copied from column A of Vo, and they are recorded near the edge of the 
strip in the proper position to match the factor matrix of Table 3. The 
cross products, Ута, are then computed. In doing this, one places the 


strip 
Table 5 
Example 1. First Trial 
A-Plane Matrix Vi 
1 2 3 4 5 6 $ 8 9 
5 (=н! Mt 
І 
II 
III 
I 
п 
II 
I 3.41 îê 
IL 74 06 
II .82 —.65 


Coefficient b 


i 
Ane 


adjacent to columns I, II, and III of Table 3. The 
corded in column 1 of Table 

In the same manner the sum of the squares of the six f 
each column of T'able 3 are computed and rec 
Table 5. 

The values of s are determined by equation (EO. 
ally, merely dividing the entry of the first e 
column. The ratio s is recorded in c 


cross products are re- 


actors loading of 
orded in the second column of 


This is, computation- 


olumn by that of the second 
olumn 4, 
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In column 3 are recorded the direction cosines of the trial reference vector 
for the A-plane from the first column of Ap. 

Column 5 is self-explanatory. The entries of column 6 are determined by 
equation (29), and they are computed from columns 3, 4, and 5. 

Column 7 is computed by equation (40). These entries are merely the 
sums of the factor loadings in the three columns of Table 2, the summation 
being taken only for the subgroup. 

The entries of column 8 are determined by equation (39). The coefficient 
b is computed as shown in the fourth section of Table 5. Its value for the 
A-plane is —.0159 as shown. 

The entries in column 8 are normalized to give the direction cosines of 
column 9. These values are also recorded in the new transformation matrix, 
which is denoted A, in Table 5. Matrix C is also computed as shown, and 
it is seen that the three reference vectors are independent, since none of the 
side entries approach unity. In higher dimensions this fact is not a guaran- 
tee of linear independence, but it is usually a safe assumption. 

The same computational procedure is followed ‘or the adjustment in the 
normals for the B-plane and the C-plane, as shown in the corresponding 
sections of Table 5. The given factor matrix of Table 2 is now multiplied by 
the new transformation matrix, A; of Table 5, and the result is the new 
oblique factor matrix V; of Table 5. The frequency distribution of factor 
loadings in each of the three columns is shown in Table 5, and a subgroup is 
indicated in cach distribution. It is a matter of judgment how large to take 
üsubgroup. If it is taken too large, the corresponding factor loadings cannot 
be made vanishingly small. If it is taken too small, an indeterminate solu- 
tion may be obtained which is revealed in the final graphs. In the present 
case the lowest six tests in each distribution were taken to constitute the 
subgroup for the next adjustment. It is, of course, not at all necessary to 
have the same number of tests in each subgroup. The size of the subgroup 
usually varies from one plane to another, and it depends entirely on the 
factorial compositions of the several tests in the battery. 

In Table 6 we have the same type of computations for the next adjust- 
ment of the reference vectors, and it is seen in the frequency distributions 
of this table that the reference axes have been so adjusted that all the tests 
in each subgroup have projections that are close to zero for their respective 


reference axes. 

The next step is to plot the three graphs AB, AC, and BC. These are 
shown in Figure 2, and it is seen that the positive simple structure is here 
evident and is complete in all three of the common-factor dimensions. These 
graphs are plotted from pairs of columns of the matrix V» of Table 6. One 
More adjustment of the reference vectors would bring the configuration 
even closer to perfection, but the appearance of the three graphs hardly 
justifies any additional refinement. 
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Numerical example No. 2 

The first numerical example started with three trial vectors that consti- 
tuted a good choice so that the simple structure was readily identified with- 
out graphical assistance except for the final set of three graphs in Figure 2 
by which the structure was accepted as complete in all three of the dimen- 
sions. In the present example we shall start with a bad choice for the third 


T'able 6 
Example 1. Second Trial 
A-Plane Matrix V, 
1 2 4 4 5 6 7 8 9 A B с 
(1—3) Ant Nme Amu А 1 04 B 
I 1.00 5 
П 1.04 4 
II1 93 5 
6 
f 
B-Plane 3 
—— 10 
—.004 216 —.00 1.00 AG i 
П | — 1027 65-103 1 67 22 
ш 040 60 57 23 
а = 24 
C-Plane 
E 
II 
ш | — 
bi 
m 7 
хте 3 
EM 36 s 4 
.0002 -0010 3 
л 
A: МА» +0 
а = :0 
A B с 4 B c wer 
I A 1.00 — ло — 
II B | — 110 99 — 
III 


С | = -15 = 29 100 


trial vector C in order to illustrate how to alter the subgroups with graphical 
aids so as to arrive at the desired planes with a minimum of compt 
It will be seen that, even with a bad initial choice of trial vector. 
solution ean be found. 

Instead of starting with the test vectors 1, 2, and 3, as in the p evious ex- 
ample, we start here with the test veetors 1, 2, and 5. At the top of Table 7 
the factor loadings of these three 


vectors are listed 
matrix Ao we have the same three vectors normalize 


itation. 
, the desired 


‚ and in the adjacent 
d to constitute the ini- 


9 о 
е 
7 
оо 
1 
o о 
l- d o 


FIGURE 2 
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tial reference vectors Ay, Ap, Ac. In the next matrix, C, we have the nen 
of the angles between the three trial vectors. Неге we пое at t ст 
the cosine of the angle between A and C is .7 7, which is unusually high, In 
the matrix V, we have the projections of the test vectors on the three trial 
reference vectors, and we also have in Table 7 a frequency distribution uit 
the projections. In columns A and B the lowest six tests were chosen for 
subgroups, and in column C the lowest Seven tests were chosen as a sub- 
group. 


Table 7 
Example 2. Initial Vectors 
Trial Vectors Matriz А. 
Test 1 Test 2 Test 5 Ал Ав Ас Aa AB Ac 
1 .65 .74 .88 I .66 75 .88 Aa 1.01 .25 UT 
П | —.67 „58 —.40 П | —.68 .54 —.40 An .25 1.00 .35 
III .33 .37 —.24 III .33 .38 —.24 Ac 7T 35 .99 
SES n! "t B 


Matriz Vo 


A B 
1 99 25 9 
2 .25 9% `8 
3| .24 35 UH 
4 -76 .В1 .6 
5| .77 135 Б 
6 -24 81 d 
T .93 .60 E: — 5 1 
8 16 93 .2 6 1 
9| .91 '81 E 
10 .50 .33 `0 
11 .23 91 ae 
12 ‚25 66 
22 68.84 SSS 


The computations of this example were made without plotting pairs 
of columns of Vo, but in practice this would be preferable. In Figure 8 we 
have two of these plots, namely, AB and AC’. In these plots the subgroups 
A and B have been circled, and they agree with the subgroups chosen in 
the frequency distribution of Table 7. In the diagram АС we would ¢ 
the subgroup that has been circled. It is not the same as the 


chosen in the frequency distribution for C in Table 7. If the subgroup had 
been chosen from the plot of Figure 2, then the solution would have been 
obtained in the present example in one less trial. Ignoring Figure 3, we take 
the subgroups marked in the frequency distributions of Table 7 

The computations for the first trial are shown in Table 8, 


hoose 
subgroup 


and they were 


FIGURE 3 


Table S 
Example 2. First Trial 
A-Plane Matrix Vi 
1 2 8 1 5 6 9 
| Sm Ж ме ЯК 


І m 66 
I1 4 GS 
In [- E X 


AT 
so 
E 


.60 mi 
08 — .68 
98 —.62 


E 1.5141 
41.6285 


2 1.01 
B “02 
Ü 36 —. 


Wi 1 
te 1 1 
ч 4 2 3 
i 1 
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carried out according to the plan described for the first ех 
was a new set of trial reference vectors, as shown in them 
gular separations are shown by the cosines in the matrix 
of the test vectors on the new trial reference vectors 
lique factor matrix V;, and the frequency 
are also tabulated. At this point graphs 
the graph for columns A and C of V,. The subgroup A is the same as in the 
frequency distribution of Table 8, but the subgroup С, which is circled in 
the graph, is not indicated by the distribution. 
from Figure 4. It consists of the tests 2, 8, 22 


ample. The result 
atrix Ат, whose an- 
C. The projections 
are shown in the ob- 
distributions of these projections 
are made, and we show in Figure 4 


A new subgroup (' is chosen 
; 4, 7, and 1. With these sub- 


FIGURE 4 


groups we proceed to the second trial With computations as shown in 
Table 9. 

The result of the second tria 
of trial vectors in the columns 
shown by their cosines in matrix 
these trial vectors are shown in t 
we have the frequency distributio 

A plot of columns A and C of V. 5, where the subgroup 
C has been circled. It is the Same subgroup as determined in the previous 
trial of Figure 4, and hence the same subgroup C is retained for the third 
trial with the computations of Table 10. Note that the s values are all small 
except for the reference vector C, which has an appreciable adjustment їй 


1 computations of Table 
of matrix As, with angular separation 
C. The projections of the test vectors 
he oblique factor matrix V, 
ns of the projections, 

21s shown in Figure 


9 gives a new set 


as 
on 
› below which 
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this trial. A plot of the pairs of columns of V; in Table 10 shows the same 
structure as in Figure 2, which was accepted as final in the first numerical 
example. Note also that the matrix A; of Table 10 is practically the same as 
the matrix As of Table 6, which terminated the first numerical example. 
This second example has been arranged primarily to show how the sub- 
group can be altered by graphs, which may be plotted for each trial. This 


Table 9 
Example 2. Second Trial 
A-Plane Matrix Va 
“al = оов 
п 035 
1 ШЕП 
B-Plane 
ү | аб —.00 1.00 
II 001 165 —,03 1.02 
UD | lbi тї 060 205 
C-Plane 
I W 9 
и | = aS 
nt 112 


approximation ап be used for factorial problems in 
e manifold is not imposed. The principal 
reason why the computational labor is reduced is that each trial vector is 
adjusted in all the dimensions of the common-factor space in each trial in- 
stead of being adjusted in only a few dimensions from the graphs in each 
trial. Hence the total number of trials is reduced. The successful selection 
of the subgroup of test У setors which is to determine each hyperplane is 


Method of successive : 
Which the assumption of a positivi 
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best made by graphical inspection. It must also be remembered that the 
final acceptance of a simple-structure solution for a test battery depends on 
the appearance of the graphs. The various criteria by which 
are interpreted have been discussed in a previous ch 
and factor patterns, 


a set of graphs 
apter on configurations 


The method of minimizing weighted sums 
Theory of the method 


One of the simplest methods of solving the rotation 


al problem in multiple- 
factor analysis is that of locating e 


ach reference vector so that the absolute 


Moun, 
Sum of test Projections is 


S ¢ ing 
hyperplane that is ddl ШШ for those 
y ae 


e 
ж „пеат th 
Were known bo tests that lie in о! ^d test? 
ог £ 'oup 

tor. If that grou] мей, 


ч. re erence vec 
е Yotationa] prok 


ijy SO 
i e desi blem would be easily 5 
Mation, o 


cessive а; s 2 
'ox Sir x “ety, Suc 
Pproxj ed solution 6 ith st 


S an also be reached W 


^ 3 ene "d 
T Space and € Vector A 


ily iP 
this tri; at the pro; j 
primary refers US trial Vector н е Proje 


qis located quite arbitrar It is 


ctions vj are computed. 


tions that the iny bi ‘or. Here ез езу steps to the position ae 
" Shall dig А ^q assu 
erence axis ig bip tor can differentiate between two 485 ue 
E n smar Я 
not affected by hag , . “ely, that the nearest primary . 


" 5 
Tong assumption. Sitive i 


ome ! 
as to whet 


he computational oute е 
y vor 
her the nearest primary 
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ence vector is bipolar or positive, but the solution is reached more quickly 
if the assumption is correct in this regard. If the original correlations are all 
positive or zero, it is a fairly safe assumption that the structure is a positive 
manifold. [f the original correlations are both positive and negative, then 
some of the primary reference vectors may be positive while some are bi- 
polar, or they might all be bipolar. 


Table 10 
Example 2. Third Trial 
A-Plane Matrix V3 
1 4 5 6 7 8 9 A po È 
GU E QA Ker Xe ker en 1 
1 017 .00 3 
I 010 101 4 
IIL 009 .02 5 
6 
: 
B-Plane 
I .006 3.41 j .00 1.00 
I 190 7 —.02 1.01 
п 4 157 .02 .09 
C-Plane 
I Bal 46 — .00 1.00 
11 p- 24 
ИП j —.00 


Coefficient b 


B 
ХХА — .0020 8 
Ye 5036 5038 
— .0001 .0021 
MAi C TEMEN 


Let it be assumed that the projections v;, cover a wide range of both posi- 
tive and negative values. If it is assumed that the nearest primary reference 
Vector is positive, then we should want to move the trial reference vector so 
that, in general, the negative projections wil! rise toward zero, so that the 
Small positive projections will fall toward zero, and so that the high positive 
Projections will remain high, or rise still higher, to identify the factor. This 
adjustment would be accomplished by moving the trial vector A, so as to 
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increase the algebraic sum of the projections except for the small 
projections which should be decreased. | m 

Let it be assumed, next, that the nearest primary reference vector is bi- 
polar. Then we should want to move A, so that the strong positive projec- 
tions rise and the strong negative projections fall, thus identifying the tests 
with high variance on the factor. The small projections should move toward 
zero. Here we want A, to move so as to let all projections rise except the 


small positive and the strong negative ones, which should decrease algebrai- 
cally. 


)ositive 


If it is desired to locate a residual plane, then the trial refere 
should be adjusted so that all its projections 
tive projections should fall, 

These three 


nee vector A, 
approach zero. Then all posi- 
and all negative projections should rise, 

cases can be summarized for convenience in Table 11. The 
positive and negative signs in this table indicate for each case whether the 


Table 11 


For Locating 


| For Locating | For Locating 
Values of 


Positive Bipolar Residual 
Projections Reference Reference Reference 
Vector Vector Vector 
rong positive, . т | + | T 
Small posi = | == 
Small negative + Ps T 
Strong negative... .. | e = | + 
adjustment of the trial vector shou 


d be made so as to m 
or decrease algebraically, 
wbitrary trial vector A, of 
Xma: The subscript » refers to the ax 
reference frame. The test vector J has the dire 
projection vj, then becomes 


ake the projections 

vj, vise or fall, i.e., to inerease 
In Figure 6 we have ; unit length and with di- 
rection cosines is of the fixed orthogonal 


ction numbers Qin, 80th 


at che 


- 
(45) rj, = N ајә 
— 


m 


The algebraic sum of the projections of all the n tests on A 


(46) Sy = NS Е 


i= 2 № > a 
کے کے‎ 


a then is 


jms 
m 1 


Let one of the test vectors be denoted J, 


vj, ON Ay, as shown. Let K be anothe 
îja On A,, as shown in the figure. If it is desired to mo 
projections approach zero, then it is 
crease v;,. The same object can be 


as in the figure, v 


vith projection 
r test vector with negative projection 
ve A, so that both these 
necessary to increase Urq and to de- 
attained by dealing with J and =K іп 
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which the direction numbers акы have been reversed. Then the desired ad- 
justment of A, is such as to decrease both vj and гы. Let us denote the vec- 
tors whose projections are to be increased or decreased by w;/J, where w; 
takes the values +1 or —1, depending on whether the test vector is re- 
versed. 

Further, let ¢ denote the algebraic sum of projections of all the test vec- 
tors ij] on A,. This sum is the criterion which is to be maximized or mini- 
mized as the case may be. Then we have for this criterion 


" A AS 
(47) фь = > Ang № 0, 
m j 


where the subseript p defines the hyperplane Ay, which is to be located by 
adjusting the trial vector Ag. 


FIGURE 6 


al veetor can be made in the direction deter- 
of change in the criterion p. This is deter- 
with respect to the direction cosines 


The adjustment of the tri 
Mined by the maximum rate 
Mined by the partial derivatives of $p 


Ang. We then have 
(48) дф» _ ҹу Wjljm = lus 
дм, F 


ection numbers of a vector L, as shown in the figure —— 
niq ате merely the sums of columns of the original 
ative signs as defined by w;. The ad- 
ane of A, and L, either toward 


Where lma are the dir 
The numerical values of l 
factor matrix F with positive or neg 
Justment of A, should then be made in the pl 
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= 7 3 "nr "s ۰ larre a isplace- 
Lor away from it; but we have, so far, no measure of how large the displ 


E be in this plane. 
o es eee a жы anim method of determining the amount 
of displacement of A,, we determine the vector E in the plane of Ay and L, 
where E is orthogonal to A,. Then E is evidently a linear combination of the 
given trial vector A, and the vector L. We then have 


(49) B= T= s 


where the parameter s is to be determined. Since A, and £ 


tare to be orthog- 
onal, we set 


(50) AE = 0 


or, by (49), 


(51) AL — зА) = 0, 


and hence 


(52) ML = s4, 


But since A, is a unit vector, we have simply 


(53) 


so that s is numerically determined as merely the scalar product of A, and 
L. By (49) and (53) the vector К is then located with direction numbers 
ёз. Let C be defined as the vector E normalized with direction cosines 
Сла. We then have A, and C as two orthogonal unit vectors. 

The projections iq аге Next computed from 


& = 
(54) Ci = S аһа 


т 
and a plot is made of jq against Ce 

is determined. When A, has been adjusted to a n 
be regarded as a trial vector to be adjusted. 
primary reference vector ean be located approximately in five 
ments of this kind. The fine adjustments should be 
plots of pairs of columns of Vip, Which are the с 
matrix V. The present method will loc 
primary reference vectors, while some 
by the eomplete set of diagrams 
seems to save a great deal of | 


From this plot the displacement of Aq 


ew position, it can again 
It has been found that each 
or six adjust- 
made by a whole set of 
lements in the oblique factor 
ate some, or perhaps most, of the 
of them may need final adjustment 
from the columns of V. The present method 
abor in solving the rotational problem. 
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Outline of computations 

(1) When a correlation matrix has been factored into a factor matrix F, 
the rotational problem can be solved by the present method. A useful way 
lo start is to select a test vector. Normalizing this test vector gives the unit 
trial vector A,. 

(2) Find the test projections v; on A, and tabulate them. 

(3) Place a strip adjacent to the column of projections vj and mark on 
this strip negative signs for all tests which are to be reversed in sign ac- 
cording to the plan of Table 11. In the interpretation of this table one must 
make some decision as to what are to be regarded as large or small projec- 
tions. In the first few trials these bands should ordinarily be rather wide, 
but in the last few trials the bands may be narrower so that a projection is 
regarded as small if it is numerically smaller than .20. 

(4) Place this strip of signs adjacent to each column of F and determine 


the weighted sums (equation [48]), 


So. = 
Wjljm = Uns + 


j 
(5) Find the sealar product (equation [53]). 


= 
gee У Meals 
e 


(6) Find the direction numbers Cma of the vector / (equation [19], 


E = 1 = shg. 

(7) Normalize E to find direction cosines Cn, of the unit vector C. 
(8) Cheek orthogonality by the relation А, =0. 
(9) Find projections cj» of the test vectors on C. 
(10) Plot v, against су. 
(11) Locate a new trial vector as 
the two-dimensional rotations. 
| (12) Find the projections on the ne 
if Necessary, 

In the interpretation of Te 
Positive or negative signs to the 
Vectors that are already in the hyperpl 


Or near the hyperplane are then given weights, w 
o namely, +1, —1, о! 0. | 
s are to be located, it is probably best to 


axes as trial vector. When a primary 
ient to choose a normalized test 


a linear combination of A, and С, as 


w trial vector, and repeat the process 


able 11 it may be useful occasionally to assign 
extremes and to ignore some of the test 
ane. These test vectors that are in 
5-0. When this is done, 


lere are three weights for tis 
When one or more residual plan 
Choose one of the last few centroid 


reference axis isto be located, 16 is conven 
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vector as the trial vector. The trial vector for the next plane should be a 
normalized test vector chosen from the tests that have near-zero projections 
on the reference axes that have already been determined, In this manner 
one can avoid starting with a trial vector that moves into the position of 
one of the reference vectors already located, 


Numerical erample 

In order to illustrate the method we sh 
reproduced from “Psychometric Mo 
factor matrix for twenty-one tests 
by the centroid method. 


all use actual data. Table 12 is 
nograph," No. 2, page 91.* This is a 
and eight orthogonal factors, computed 


Table 12 
Factor Matriz Fo 

T Il III Iv ۴ VI VII VIII Ud 
1 51 —.27 3 3 —.09 —.25 ; 
2 .62 —.31 06 ; de —27 2o 
3 -63 —.09 —.09 g .08 —.21 
4 47 .28 14 33 —.17 A7 — 
5 .38 .06 —.14 Б 10 —.11 ~ 
6 .40 .19 —.19 —.84 —.28 14 — 
a -66 -40 .22 —.10 —.09 
8 .69 .43 .23 —.19 — 09 
9 .70 .28 28 —.10 -—119 — 
10 .63 .26 =; .19 (4 —.17 – 
11 .59 .22 = .23 Jf =.03 = 
12 -53 +29 = .20 —.15 —.09 
13 -48 —.44 = 16 .16 16 — 
14 .40 —.50 = 17 .18 11 
15 .46 —,47 =, 12 «12 08 
16 -57 —.26 —.05 —.23 17 — 
17 .58 —.21 —.08 —.94 18 — 
18 .64 —.9] 04 .03 18 — 
2B 188 13 ‘ 168 .29 19 

.65 .23 .06 seva .28 £ 
21 .65 04 —.16 —.10 .26 09 


Trial 1 is represented by one of the 
one of the test vectors, chosen arbitrarily. This is test 14 in the 
battery. Its direction numbers are recorded in the first column in Т 
This vector is normalized by the multiply: 
tom of the first column, and we then have the direction cosines of the first 
trial vector, A, as shown in the next column, The Projections of the test 
vectors on this trial vector are listed in column V, trial 1, of Table 14. Here 
it will be assumed that the structure is a positive manifold, and hence the 
primary reference axes are assumed to be positive rather than bipolar, Ac- 


sections of Table 13. We start with 


present 


‹ able 13. 
Ing constant, as shown at the bot- 


* L. L. Thurstone and Thelma Gwinn Thurstone, Factorial Studies of Tntelligence 
(Chicago: University of Chicago Press, 1941), 
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cording to Table 11, we should reverse the signs of the test vectors with small 
positive projections in this column V. We do this by reversing the signs of 
all test. vectors whose positive projections in this column are less than .20. 
The reversals are indicated by the negative signs in the column adjacent to 
the projections v. 


Table 13 


Trial 1 Trial 2 
Ps \ 1 З Re \ L e c 
Sine 3 —2.20 
{кы —1.10 
111 y .86 1 
IV = 61 IV 
ү 64 \ 
ME 65 \ 
ҮП | .33 Y 
VIII | .05 VI 
Norm 1 ‚310009 Norm  .985664 .460092 
S |3 S . 9793 
| 
Trial 3 Tris 
| A a È » - Ne A L e с 
OS ap К тас EE TE 21 — .25 4.96 .94 
k 725 26 — .0 —›86 == „85: .00.—.08 
111 ye Es Du. 64 — .24 —1.11 —.21 
Me М = > —.37 68. 1.18 .22 
IV |= —.30 —1.16 — = 98 97 — И —.02 
vi :25 20 2 10 33 19 04 
vil | o3 M- 09 40 28 105 
vin | 9 09 07 67 57 11 
Nor cm ‚190052 
1.2495 


ee 


С 381954 


992925 
2.8184 
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These negative signs are now copied at the edge of a strip which is placed 
adjacent to each column of the factor matrix in Table 12, and the weighted 
sums are determined. These weighted sums are recorded in column L of 
Table 13, trial 1. The scalar product s is next computed and recorded as 
shown. This enables us to compute the direction numbers e of the vector E, 
as shown in the next column. The normalizing constant for this vector is 
shown at the bottom of column e, and then we have the direction cosines 
of the unit vector C, as shown in the last column for this trial. It is useful to 
check the computations to make sure that the vectors C and A are orthog- 


onal by computing their scalar produet, which should vanish. 
Table 14 
Талі 1 Triar 2 TRIAL 3 Tria Tran 5 
TESTS E ES z= е » RE 
[^ Ү + © V t Y + E ۷ + (^f V + 
1 - d£ ——.42 .12 ——.80 206 — .88 =.01 .03 — 
S в аб SAS .43  —:29 BB 56 .30 34 
3 - 26 чой: .95 ——.95 B. GE 3 = aT 
4 = .05 — —.34 —.04 .17 —.11 .39 —.06 —.03 
5 - x» б LOL Iy 20° 216 = 18 
6 – 09 — —.22 (01 — 95 —:02 :20 03 = 05 = 
7 -= 04 ——.24 —.07  —:02 —:10 .68 —.11 —.05 
8 - :05 — —.18 —.04 .10 —.10 .66 —.08 02 
9 - Aah SG) Oe ge др = 48. y = .16 
10 .11 — —.26 — 01 .04 —.04 .56 —.05 =< Gt 
ll 62 7-3 70 = 03.08 = .52 02 — .06 — 
12 —.65 -04 — —.13 – .07 11 —.08 .48 —.07 —.03 
DE O жэл ч, эсу р -36 .67 10 
14 01 .8 =.02 вз _ 05 .83 28 0 82 
ly Ol ea’ gg: Vey с 03  .79 8. 177 .80 
I8. eT Дед 208 осо .01 = .64 —.04 JI e 
17 —.52 .08 ——,80 — 91 С 28 —.08 .68 —.14 —.09 
18  —.88 .33 м7 ‘9g __ 32 .19— .67 (12 — 17 
Ig = 0% ву YS Sue 44 .07 — .73 – 02 .03 
20 —.33 .18 ——:56 пг 32 .03 — .66 — 03 02 = 
2l 484 2h 06у ^19 .— 43 .09 — 69 00 .05 — 


The projections of the test vectors on the unit correction 
next computed and recorded in column C, trial 1, of Table 14. 

The projections v are then plotted against the projections с, as shown in 
Figure 7. The new plane is located as shown in this graph by its trace 
through the origin. The normal to the trace defines the new extended trial 
vector, A.. This new trial vector is а linear combination of the first trial vec- 
tor, A, and the correction vector, C, The vector equation for the new 
vector is shown in each graph. The parameter н 
mined readily when the plot is made 
tions of the cross 


vector C are 


trial 
18 in this graph is deter- 
On cross-section paper. The fine grada- 
-section paper are omitted in these diagrams 
The direction numbers of the new tri 


al vector are recorded in the first 
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column of the next section of Table 13, trial 2. This vector is normalized 
by the normalizing constant at the bottom of this column, and we then 
have the new unit trial vector, A, whose direction cosines are listed in the 
next column. The notation A, represents the new trial vector, but it is longer 
than a unit vector, The first step in each trial is to reduce the new extended 
trial vector to unit length. 

The caleulations proceed here as before to the determination of the cor- 
rection vector C for trial 2. The projections of the test vectors on the trial 
vector A and on the correction vector C are computed and recorded in 
Table 14, trial 2. The plot is shown in Figure 8. The small positive projec- 
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Figure 11 


tions on the new trial vector which are to be reversed in sign for the weighted 
summation are shown in black on the previous graph, Figure 7. These re- 
versals can be anticipated before the new trial vector is determined. 

This procedure is repeated, as shown in Figures 9 and 10, until the fifth 
trial, which produces the graph in Figure 11. Since this graph does not indi- 
cate any improvement to be gained by moving the trial vector in the plane 
of A and the correction vector C, we accept the trial vector A for this trial 
as a close approximation to the desired reference vector. 

It ean be scen in Figure 11, as well as from the last column of Table 14, 
that there are three tests which are exceptionally high on this reference 
vector. These are the three space tests of the test battery in this experi- 
ment. In order to locate another primary reference vector, we should start 
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by normalizing one of the test vectors, which have small projections as 
shown in the last column of Table 14. | : 

'The same procedure was used to locate the residual plane in this test 
battery. This was done by starting with the last centroid column of Table 19 
and by reversing the signs of all test vectors which had positive projections 
on the eighth centroid axis. This leads to an improved position for the resid- 
ual reference vector. One trial with this method was sufficient to locate the 
residual reference axis for these data. 

When the primary reference axes, as well as one or more residual refer- 
ence axes, have been determined by this simple method, the final step is to 
plot the whole set of diagrams, one diagram for each pair of reference 
tors. Since there are eight dimensions in this problem, the final set of 
grams will have twenty-eight graphs. Inspection of these graphs e 
one to determine any further small adjustments that m 
locate the best position for the simple structure. The present method should 
be regarded as adequate for obtaining an approximate location of the refer- 
ence frame of a simple structure; but it should always be followed by a com- 
plete set of plots of pairs of columns of the oblique factor matrix, to make 
sure that the best location of the frame has been attained. The procedure 

here described is perhaps the simplest of the rotational methods so far de- 
vised, and it should serve to eliminate a very large part of the labor which 
has hitherto been necessary in locating a simple structure in multidimen- 
sional factor problems, 

In the application of this method or simil 
problem, it will be found that some configurations are dominated by the 
large variance of the first centroid as compared with subsequent factors and 
that this circumstance causes slow convergence in these solutions. In deal- 

ing with such configurations it may be found useful to extend the test vec- 
tors as in the method of extended vectors. The configuration is then unal- 
tered except for the fact that all the test vectors are extended so as to have 
unit projection on the first centroid axis. The present method can be ap- 
plied to such a configuration in the same manner as here described. In the 
location of the new trial vector, one has additional freedom, in that 
of points on the diagram which are not radial ¢ 
plane directly by expressing the next trial у 
three vectors, namely, the given trial veetor 
the unit reference vector along the first 
projection on the first centroid axis 
determined in the same manner 


vec- 
dia- 
nables 
ay be necessary to 


ar methods to the rotational 


a streak 
an be brought into a hyper- 
ector as a linear combination of 
› the new correction vector, and 
axis. All the test vectors hav. 
‚ and the new trial vector с 
as in the method of extended у 
procedure gives the solution in two trials with the pre: 

Another variant of the method here described is te 
co-ordinates of the correction vector in 


e unit 
an then be 
ectors. This 
sent example. 

> weight each of the 


versely as the absolute sum of the 
entries in the corresponding columns of the given factor matrix F. The 
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same result can be attained by writing a new factor matrix in which the ab- 
solute sums of the columns are equal. This involves a distortion of the con- 
figuration, but the simple structure is nevertheless invariant under this kind 
of distortion. The disadvantage of this procedure on experimental data is 
that the axes on which the variance is very much extended become so dis- 
torted that the residual variance is more difficult to differentiate from the 
significant variance which is to be interpreted. 


A double-group method of rotation 

It happens not infrequently in a plot of two columns of the oblique factor 
matrix that the points reveal a concentration which could be fitted by a line 
that docs not pass through the origin. In previous chapters there have been 
a number of such cases. A few examples will be listed here to show the situa- 
tion in which the double-group method is applicable. Figure 3 of chapter x 
shows three such possibilities in the same diagram. In that figure there isa 
roughly linear arrangement of points that could be fitted by a straight line 
through points 1 and 3. A second line could be drawn through points 2 and 
3, while a third line could be drawn through points / and 2. All but two of 
the points in the figure would be near one or more of these three lines. In- 
stead of locating the line through points / and 2, we might choose the cen- 
troids of the two groups, 1, 13, 18, and 8, 15, 20, which would, in this case, 
give practically the same solution, 

In Figure 5 of chapter x we have another case. The two groups, 1, 13, 18, 
and 8, 15, 20, might be chosen in this diagram to define the next adjustment 
in plane C. In Figure à of this chapter the plane C might be defined by a 
vector that is set orthogonal to the two groups, 2, 8, and 7, 7. 

The principle involved in this adjustment is to express the new vector Ai 
as a linear combination of the given unit vector Ao and the two centroid vec- 
tors C, and (Cz of the two groups chosen graphically. We start, then, with the 


following notation : 
Ay = given unit reference vector , 
A, = new unit vector to be determined , 
C, = centroid vector for group 1, 
C, = centroid vector for group 2, 
Li = С, where І = 1, 


» where L2 = 1. 


1% = К 


The vectors Lı and La are collinear with the corresponding centroid vec- 
tors. The new vector A, 15 expressed as the linear combination 


(55) Ay = Ao + pila + pola, 
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where p; and рз are scalars to be determined. The vector A, must s: itisfy the 
two requirements of orthogonality with Ly and Ls. Then we have 


(56) Ail, = 0 
and 
(57) Ам» = 0. 


Substituting the linear combination for Аз, we have 
(58) (Ao + pila + pals) = 0, 
from which we get 

(59) Aola + pil? + polls = 0. 
Ly is a unit vector, and hence 

(60) Bel, 

Let Aoli = s; and let Lila = l. Then 

(61) sit pit pl = 0. 

For the second condition of orthogonality, wè have, similarly, 
(62) (Ao + pila + ра) = 0, 


from which we get 


(63) sa + pil + ps = 0 
where $$ = Aola. Solving for De, We get 

j sb йв 
(64) аер 
and, by analogy, 

T" Sol — sı 

6: RII аад 
(65) fia 


When the parameters Dı and ps are 


r new vector 
Ai can be determined. It is normalized | to become the new unit reference 
vector, Ay. When the double 


a ically indicated, it is 
quite effective, 


how 


CHAPTER XVIII 
SECOND-ORDER FACTORS 


First-order and second-order factors 

Most of the work that has been done so far in the development of factorial 
theory has been concerned with the factors obtained from the test correla- 
tions with or without rotation of axes for the selection of a suitable reference 
frame. Factors that are obtained from the test correlations will be called first- 
order factors, whether they are selected so as to be orthogonal or oblique. 
We shall now consider the factors that may be determined from the correla- 
tions of the first-order factors. actors that are obtained from the correlations 
of the first-order factors will be called second-order factors. Vactors of this 
type seem to be of fundamental significance in the interpretation of cor- 
related variables. 

Analysis of second-order factors and their relations to those of first order 
can be presented in several different ways. We shall describe these two 
factorial domains in terms of a literal notation, a physical example, a dia- 
grammatie representation, а geometrical example, and the matrix equations 
relating the two domains. 

Consider, first, à reduced correlation matrix for the tests whose rank is, 
say, 5. The factoring of this correlation matrix determines five arbitrary 
orthogonal unit reference vectors which may be denoted 7, IZ, IIJ, IV and 
V. This orthogonal reference frame is arbitrary in the sense that it is de- 
fined by the method of faetoring whieh happens to be used, This reference 
frame will be regarded as fixed, and all other vectors will be defined in terms 
of this fixed orthogonal frame, which is designated by the subscript m. Let 
it be assumed that a complete simple structure can be found in the test eon- 
figuration and let the corresponding primary vectors be denoted 4, B, C, 
D, and E. (‘These are ordinarily denoted Ta, Ta, ete.). We shall assume 
that these primary traits are correlated in the experimental population. 
Then the primary vectors in the test configuration will be separated by 
oblique angles, whose cosines are the correlations between the primary 
traits in the particular group of subjects studied. Let these correlations be 
listed in a new correlation matrix of order 5X5, showing the correlations 
between primary factors. This correlation matrix defines the second-order 
domain just as the correlation matrix for the tests defines the first-order 


domain. 
The simplest case is that in which the five primary factors are uncorre- 
lated, in which ease their correlation matrix is a unit matrix so that an 
411 
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analysis of a second-order domain is not immediately indicated, Next would 
be the case in which the reduced correlation matrix for the primary factors 
A, B, C, D, and E is of unit rank. There аге two types of interpretation for 
such a situation. The correlations between the primary factors in a particu- 
lar experimental population may be due to conditions of selection of the 
subjects, and in this case the correlations would be of no more theoretical 
importance than the conditions of selection of the subjects. If, on the other 
hand, the five primary functions, A,B,C, D, and E, actually do have some 
parameter in common, then one would expect their intercorrelations to be 
of unit rank for different experimental groups of subjects that are selected 
in different ways. In other words, the mere fact that a set of variables or a 
set of factors is correlated does not imply any scientific obligation to find 
"the" factors that account for the correlations, because the factors, if found, 
might turn out to be as incidental in significance as the conditions by which 
the subjects happened to be selected. On the other hand, the fact that corre- 
lations between variables, or between factors, can be caused by scient ifically 
trivial cireumstances does not guarantee that all correlations between vari- 
ables are of trivial significance. If the correlations between the five primary 
factors in the present example should turn out to be of unit rank, then this 
circumstance merits a closer look because such a simplification would not 
often happen by chance. If the correlations between the primary factors 
should turn out to be of unit rank for several different experimental groups, 
then we should have an obligation to ascertain the cause which must tran- 
scend the selective conditions. 

Let it be assumed that the five prim 


ary factors do have a parameter p 
in common. Then the five primaries cou 


Id be expressed in the form 
A = f(p, a), D = f(p, d), 

B = f(p,b), 
C = f(p, с), 


E = f(p,e), 


as a, 
rameter, p, which 
s. If there should 
een the parameters a, b, c, d, 
en the unit rank of the second- 
order domain would be disturbed. If the correlations of the primaries show 

the parameters a, b, c, d, and е, а second- 
order parameter or factor p can be postulated, It should be noted that we 
now have six parameters, namely, a, b, c, d, e 


e, and p; and, since the rank of 
the test correlations is 5, it follows that these six parameters are linearly de- 
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pendent. In fact, the parameter p is now a linear combination of the other 
five parameters. We can express these relations by a set of parameters such 
as A, B, C, D, E, and p, in which pis a linear combination of the five primary 
parameters. The five primaries are parameters descriptive of the first-order 
domain, and the parameter or factor p is descriptive of the second-order 
domain, which is of unit rank here. The second-order parameter is a linear 
combination of the five primaries that are defined by the original test cor- 
relations. If some degree of consistency can be found for these parameters 
for different groups of subjects, then all these parameters should represent 
some aspects of the underlying physical or mental functions. 

Consider, next, a set of correlated primaries, A, B, C, D, and Е, in which 
the parameter p appears in the first order, as in the following example: 


A = fly, a), D = fip, д), 
В = Јр, b), Е = fip). 


С = (руе), 


The rank of the reduced correlation matrix of the tests would now be 5. 
The five primaries listed above would be correlated and of unit rank. The 
second-order factor p would be determined from the correlations of the 
primaries. In this ease the communality for the primary factor 2 would be 
hear unity, thus showing that its total variance is common to the second- 
order factor p. Henee the primary factor Û of the first order and the factor 
p of the second order would be identical. The presence, or absence, of the 
primary / could be determined by including, or excluding, a few tests in 
the battery. We sec, therefore, that the appearance of a factor in the first 
order or in the second order may depend on the battery of measurements 
taken. Hence a factor should not be considered as intrinsically different 
because it appears in the second order, This circumstance can be deter- 
mined by the selection of the test battery. 

On the other hand, a parameter which always appears in the measure- 
ments in association with some other function would not appear as in the 
primary JÛ, and it would be discovered experimentally, nearly always in the 
second order. Such a limitation could be introduced by the physical nature 
of the attribute which the factor represents, so that in such a case the second- 
order factor would represent something fundamentally different from that 
of the first order, A single-factor study is not likely to reveal whether a 
second-order parameter is fundamentally different from the parameters of 
first order or whether the differentiation is caused merely by the selection of 


the test battery. 
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In the following example we have another combination of primaries: 


А = Јр, а), D = Јр, а), 
B = f(p,b), E = f(e). 


C = f(p, c) Е 


In this example the reduced rank of the test correlations would again be 
5. The correlations of the primaries would show unit rank for A, B, C, 
and D. The factor E would be orthogonal to the 


its row and column would have side correl 
primaries would not be of unit rank if we consider the whole table of order 
5X5, but it would be of unit rank if we consider only the 4X4 table for 


A, В, С, and D. Relations of this kind can be found by inspection of the 
correlations of the primaries, and they may be indicative of the underlying 
order of the domain that is being investigated, 

The principles of a second-order domain have been discussed here in 
terms of the simple case in which that domain is of unit rank, so that there 
is only one general second-order factor. It should be evident that the or- 
ganization of the second-order parameters can be of any rank 
ity. For example, the rank may be higher than 1, and the second-order fac- 
tors may extend to all the primaries or to only some of them. The possi- 
bility of third- and higher-order factors must be recognized; but their ex- 


perimental identification is of increasing difficulty the higher the order, be- 
cause of the instability of 


such a superstructure on practically feasible ex- 
perimental data, The number of second-order factors that can be deter- 
mined from a given number of line; 


arly independent primary factors follows 
the same restrictive relations that govern the number of primary factors 
that can be determined from a given number of tests, Thus, for example, 
it is not to be expected that three second-order factors would be deter- 
minate from only five primary factors, for the Same reason that three pri- 
mary factors cannot be determined from five tests, Furthermore, 
tirely possible in the same data for the first-order 
pretation of a set of primary factors and for the second-order domain to be 
indeterminate or ambiguous in the same data. 

In order to avoid misunderstanding, perhaps it should be re 
in factor analysis we are using the term parameter in its universa] meaning in 
science, A parameter is one of the measurements that are used for dese "ibing 
or defining an object or event. In statistical theory the term parameter is fre- 
quently used in a more restricted sense as descriptive of t 
trasted with a statistic, which is the 
ple. We are not using the 


rest of the system, so that 
ations of zero. The correlations of 


and complex- 


it is en- 
domain to give clear inter- 


marked that 


he universe, as con- 
corresponding measurement on 


a sam- 
term in this restricted sense. 
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The box example 

In order to illustrate the nature of first- and second-order factors we shall 
make use of populations of simple objects or geometrical figures and their 
measurable properties instead of dealing with these factors merely as logical 
abstractions. We have used a population of rectangular boxes and their 
measurable attributes to illustrate the principles of correlated primary 
factors, and we can use them also for the present discussion. 

A random collection of rectangular boxes was represented by the three 
measurements, length (x), width (y), and height (z) (chapter vii). A list of 
measurements was prepared which could be made on each box, such as the 
diagonal of the front face, the area of the top surface, the length of a vertical 
edge, and so on. Each of these measurements represented a test score, and 
each box represented an individual member of the statistical population. 
The correlations between the measurements were computed and analyzed 
factorially as if we did not know anything about their exact nature. 
Each measurement was treated as a test score of unknown factorial com- 
position, As has been shown previously, the analysis revealed three factors 
in the correlations for the particular set of measurements used. The con- 
figuration showed a complete simple structure, and a set of primary vectors 
was determined by the configuration. These three primaries represented the 
three basie parameters in terms of which all the test measurements had been 


expressed, 

The three primary vectors were separated by acute angles whose cosines 
represented the correlations between the three basic parameters that were 
used in setting up the box example. These three correlations could be as- 
sembled into a small correlation matrix of order 3X3. The physical inter- 
pretation of the positive correlations was that large boxes tend to have all 
their dimensions larger than those of small boxes. In other words, if one of 
the dimensions of а box shape is, say, 6 feet, the other dimensions of the box 
are not likely to be of the order of, say, 2 or 3 inches. The table of correla- 
tions of the three primary factors, X, Y, and 7, could be represented by a 
single common factor. This factor would be a second-order factor. It would, 
no doubt, be interpreted as a size factor in the box example. If this second- 
order size factor were denoted s, we should have four parameters for de- 
scribing the box shapes, namely, the three dimensions z, y, and z and the 
size factor s. These four parameters or factors would be linearly dependent 
because the rank of the correlation matrix of the tests was 3. 

In the case of the box example, a size factor or parameter could be deter- 
mined in the first order if desired. lor this purpose we could use the first 
centroid axis, the major principal axis, or the volume vector, all of which can 
be easily defined in the first-order system of test vectors. The four param- 
eters so chosen would also be linearly dependent. If we wanted to use 
only three linearly independent parameters, including a size factor, that 
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could be done in the first order by choosing, say, the two ratios z y= rı and 
т/2 = т» аѕ well as the volume vector v. These three factors would be linearly 
independent, but they would be correlated. The latitude with which we can 
choose simplifying parameters for the box example is determined in part by 
the fact that three factors can nearly always be represented by a common 
factor, whereas this is not the case when the rank is higher than 3. 


Diagrammatic representation 


The relations between the first-order and the second-order domains can 
be represented diagrammatically as shown in Figures 1,2, : 


»2,and3. In Figure 1 
we have a set of eight tests, whose correlations are accounted for by five 


Ficure 1 
primary factors, A, B, C, D, and E 


for example, is present in the common-factor variances of lests 1, 2, 
The primary factor E is present in the common-f 
tests, and hence E would be called 


; Which are uncorrelated. The factor A, 


and 4, 
actor variances of al] the 
à general factor for the particular bat- 
tery. Since it is orthogonal to all the other primary factors, it would be 
called an orthogonal general factor of the first order, In order to det ermine the 
nature of the factor / it would be necessary to study it in different test bat- 
teries so that one could predict. with certainty when the factor would be 
present and when it would be absent from a test. Since the primary factors 
are here represented as uncorrelated, the matrix of correlations of the pri- 
mary factors would be an identity matrix, and there would be no immediate 
provocation to investigate a second-order domain, 
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In Figure 2 we have represented a set of tests and five primary factors, 
A, B, C, D, and E. (We are not here concerned as to whether the particular 
number of tests represented in this diagram is adequate for the determina- 
tion of five primary factors. The purpose of these diagrams is merely to 
show the nature of the relations between the two domains.) The rank of the 
correlation matrix of the tests would here be 5, which corresponds to the 
number of linearly independent primary factors. In the present case we 
should find that the primary factors are themselves correlated. The matrix 
of correlations of these primaries would be of order 5X 5, and it would be of 
unit rank. The correlations between the primary factors could therefore be 


FIGURE 2 


accounted for by a single general second-order factor that is denoted G. If 
both the first-order and the second-order factors were to be used for the de- 
scription of the tests and the relations, we should have six parameters which 
would be linearly dependent because the rank of the correlations of the tests 
is only 5. In fact, the saturation of each test with the second-order factor G 
would be a linear combination of the saturations of the test with the five 
primaries of the first order. None of the primary factors are general factors 
in this figure. 

In Figure 8 we have a more complex relation, in that the correlation ma- 
trix for the primary factors would be of rank 2. One of the second-order fac- 
tors is here shown to be common to all but one of the primary factors; one 
of the second-order factors is a factorial doublet, in that it represents addi- 
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tional correlation between the primaries B and D; and the primary factor A 
is orthogonal to the rest of the primaries, so that it does not participate in 
the second-order domain. This diagram is drawn merely to illustrate the 
variations in complexity that may be found in factorial studies, 

The two types of general factors here shown in Figures 1 
interesting differences. The general factor E of Figure 1 is independent of 
the other primary factors, while the general factor G in Figure 2 is present 
in all the other factors. Hence we must conclude that 


© 
9 


and 2 have some 


à second-order gen- 


FicunkE 3 


eral factor is a part of, and must participate in, the definition of the other 


factors, while the orthogonal general factor EZ of Figure 1 is, by q finiti 
independent of the other primary factors. It is evident ШЕЛ. poe 
general second-order factor is likely to be of more fundamenta] si P А. | : 
for the domain in question than à general orthogonal first-order eid aoe 
orthogonal general factor of the first order might operate in a ea e к | 
any group factor, whereas a second-order general factor would ое 
ordinarily, through the mechanism of some function that could be identified 
as a group factor, a primary factor, or a Special ability, 


The factor patterns corres i relati i 
E patterns corresponding to the relations shown diagrammati- 
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cally in these figures are given in Tables 1, 2, and 3. Table 1 shows the factor 
pattern for Figure 1. Here the orthogonal general factor Æ is identified by 
the fact that all entries of its column are filled. Table 2 shows the factor pat- 
tern for Figure 2. Here it is seen by the factor pattern that a group such as 
tests 1, 4, 5, and 7 have no primary factor in common and that hence their 
correlations would be determined only by the second-order general factor G. 
The determinant of the correlations for these four tests (the tetrad differ- 
ence) would therefore vanish. The second-order factor matrix is also shown ` 
in this table with only one factor G to correspond to this example. 


Table 1 Table 2 


Orthogonal Factors Correlated Factors 


1 B € D E 4 B e D 
AT ANC IL x TIRE 
2 x x x 2 x x x 
3 x х З x 
4 | xe X x 4 x 
5 x 5 x x 
6 x SM x 6 x М x 
7 x X 7 X 
8 x X 8 E ж 
9 X 
Table 3 ГЕ" P 
Second-Order Do- — Жасар 
main of Figures: д В 
2 Me 5 x 
Р Q D x 
———————— E x 
A | MR igo 
h x 
С 
D x 


& 
XXXXXX 


The question might be raised as to whether both types of general factors 
could be present in the same battery. That seems possible. In that case a 
simple structure could define the primary factors A, B, C, and D but not 
E in the particular battery of Figure 1. This factor could be assumed arbi- 
trarily to be orthogonal to the other factors, but then the line GE of Figure 2 
would be erased to correspond to the fact that E is orthogonal to the other 
factors. One or more second-order general factors eould be found in the cor- 
related primaries. If the correlations of 4, B, C, and D were of unit rank, 


another alternative would be to set E in such a relation to the other primary 
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vectors as to maintain the unit rank with the second-order general factor. 
It might then be found that the vector E has non-vanishing projections on 
all the test vectors, in which case both types of general factor would be as- 
sumed to be a possible set of explanatory parameters for the battery in 
question. It must be remembered that these various locations of the refer- 
ence frame for the explanatory parameters in both the first-order and the 
second-order domains have validity only in so far as they are suggestive of 
fruitful scientific interpretation. If this is not the purpose, then the factorial 
resolution might as well remain in the arbitrary ort hogonal factors produced 
by factoring the given test correlations—or, better still, by not doing the 
factoring at all. 

It might be asked how the correlations of 
into a second-order domain of unit rank which is lower than the rank of the 
test correlations. The transitions can be regarded geometrically. The unit 
test vectors usually define a Space of as many dimensions as there are tests. 
When the reduced correlation matrix is considered, its r 
lower than its order. Hence the reduction from the number of tests n to the 
number of primary factors r represents a reduction from t 
of the tests to the common-factor vari 
trix for the prim 


a test battery can be resolved 


ank is frequently 


he total variance 
ance. The complete correlation ma- 
ary factors represents a set of r unit vectors in as many di- 
mensions, the dimensionality of the common-factor space, The reduced 
form of this matrix for the example of Figure 2 would have unit rank be- 
cause the side correlations are determined only by that which the primaries 
have in common, namely, the second-order genera] factor, 


Group factors and primary factors 


In Figure д we have a diagrammatic representation of 


a different kind of 
resolution of factors in th 


e second-order domain and their relation to the 
primary factors, In this example the rank of the correlation matrix of tests is 
assumed to be five, as represented by as many primary factors, ABE D; 
and E. Let it be assumed that the correlation matrix for 

is of unit rank. The genera] second-order factor G then accounts for the ob- 
Served correlations of the primary factors, If the five linearly independent 
primary unit vectors and the second-order unit vector G are to be represent- 
ed in the same space, the dimensionality of this Space must be six, It is 
possible to locate in this augmented space another set of unit vectors, a, b, 
€, d, and e, which are mutually orthogonal and which are also orthogonal to 
the unit vector G. Then we have the orthogonal reference frame Gd, b; 
6, d, and e, which defines the six dimensions of the first- and second-order 
factors but not the test space, The five linearly independent primary factors 
define a five-dimensional space corresponding to the rank of the test corre- 


lations, and this space is a part of the total six-dimensiona] Space of this 
representation, 


these five primaries 
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The unit vector a is a linear combination of the unit vector G and the 
primary vector A. The relation is similar for the other primary vectors. The 
primary vectors A, B, C, D, and E are correlated and of unit rank, whereas the 
vectors a, b, c, d, and e are arbitrarily set orthogonal to each other. In gen- 
eral, if the rank of the test correlations is r and if the correlations of the 
primaries are of unit rank, then the primaries define a unit vector G for a 
general second-order factor in an augmented space of dimensionality (r4-1) 
and also a set of r mutually orthogonal unit vectors, each of which is in the 
plane of the second-order general factor and one of the primaries. These 
vectors are arbitrarily set orthogonal to the general second-order factor, and 


FIGURE 4 


they are called group facto In Figure 4 the primary factors are denoted 
A, B, C, D, and E, and the group factors are denoted a, b, c, d, and e. With 
this resolution we have (7+1) linearly dependent factors, which represent 
the test correlations of rank r. This type of resolution is preferred by some 
students, who use the reference frame G, a, b, c, d, and e because it is orthog- 
onal rather than the frame G, A, B, C, D, and E, which is oblique. 


A general second-order factor 

The algebraic and computational relations between the first-order and 
the second-order domains will be shown for the case of a single general 
second-order factor because of the interest of this case for the psychological 
controversies of the past forty years about Spearman’s general intellective 


factor. The algebraic and computational relations to be shown can be gen- 
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eralized to second-order domains of higher than unit rank. It must be re- 
membered, however, that the restriction of our discussion to unit rank for 
the second-order domain does not in any way imply th 
always to be expected. The methods of analy: 
a second order of higher rank when the dat 
order configuration. In any case the sec 
ably lower than the rank r of the 
pretation. 

The primary vectors constitute a set of r lines 
tors that define a space of dimensionality equal to the rank of the test corre- 
lations. In order to represent a general 


second-order factor as a unit vector 
in the same configuration it is necessary to augment the dimensionality to 
(r+1) dimensions. A second-order domain of rank 2 would thus require an 


augmented space of dimensionality (7+2). The projections of the test vee- 
tors on these additional vectors in the augmented space can, however, be 
pressed as linear combinations of the test projections on the prim 
tors or on any set of r linearly independent vectors in the c 
space. The procedures for determining these satt 
out writing explicitly the (r+1) co 
tors in the augmented space. 

The present discussion is confined to factorial dat 
ditions, namely, (1) that a comple 


at such low rank is 
can be readily extended to 
a indicate a determinate second- 
ond-order rank should be consider- 
first-order factors in order to just 


ify inter- 


uly independent unit vec- 


ex- 
üry vec- 
ommon-factor 
irations will be shown with- 


-ordinates of the second-order unit vec- 


a that satisfy tw 


о Con- 
te simple structure is rey: 


ealed in the test 
configuration and (2) that the second-order correlation matrix is of unit. 
rank. 


These methods can be adapted to the 
factors, and the methods е 


One of two objectives wi 


analysis of less th 
an be adapted to higher second-order rank. 

П be assumed, namely, (1) to determine the pro- 
jections (saturations) of the tests on the second-order factor in addition to 
the projections on the primary reference vectors or (2) to determine the pro- 
jections on the second-order factor and also on the orthogonal group factors, 
It will be convenient to discuss the algebraic relations under four case 


ause of the different computational routes that may be chosen, 
cases are as follows: 


an r primary 


s be- 
These four 


Case 1. Transformation from F to V, including the column vector G 
This transformation is shown in rect 


angular notation in Table 
equation 


4 for the 


(1) №№, = Fog 
in which the matrix V;p has an extra column for the second-order factor G 
with elements Via, which may also be denoted Tjo because these are the corre- 


lations between the tests j and the general factor G. The transformation 


matrix W,,, is identical with Amp except for the added column С with ele- 
"them 


ments Ymg, which are to be determined. Consider atrix T' as an exten- 
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sion of the factor matrix F. The rows of T give the direction cosines of the 
primary vectors T; with elements б. The same transformation gives 


(2) Tu mp = Fipa 

which is the diagonal matrix D except for the first column. Applying the 
transformation Yng, we have 

(3) Tu ing = Tu; 


Table 4 


Case 1. Transformation from F to V, Including the Column Vector G 


"E MENO 
| 
J 
Gn 
F 
Z I m 
T A 
T Lim 7 
Te t 
т 
Case 2. Transformation from F to U, Including the Group 
Factors and the Column Vector G 
"e MNT S LE ча че = ГЛИС ЯМЕ 
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where r,, is the first column of V, and its elements are the correlations of the 

99 "ү M A 
primary factors with the general factor G. These are known from the facto 
ing of the unit-rank correlation matrix for the primaries. Then 


(4) Ving = TU 


tml tg 5 


and, since T = DA~!, we have 
(5) У. = AD ry, , 


from which the first column of Wing can bec 


omputed. Hence the column G 
of the augmented oblique factor m 


atrix V becomes known. 
Case 2. Transformation from T 
factor G 
Here the computation starts again with the orthogonal factor mat rix Г, 
and the objective is to determine the saturations of the tests j with the r 
group factors and the second-order general factor G. 
also shown in Table 4 in rectangular not 


" to U, including group factors and general 


This transformation is 
ation by the equation 
(6) КыЛ. = U 


jw, 

where U is the factor matrix, showing the projections of tests j on group fac- 
tors and general factor G. These (r4-1) mutually orthogonal factors will be 
designated by the subscript w. The first column of this matrix is again the 


column of correlations tip: ansformation is applied to the ma- 


If the same tr: 
trix T for the primary 


vectors, we have 
(7) Ta uus = Uo 
which is also a diagonal matrix except for the first column, which contains 
the correlations ть between the primary factors and the second-order gen- 
eral factor G. These saturations can be determined from the unit-rank cor- 
relation matrix ТТ” = fe, for the primary factors,* Consider the first row 
of Ur. The two entries in this row show the direction cosines of T 
of the orthogonal frame G, a, b, and €. The primary vectoy 7 
combination of the two orthogonal unit vectors Gand a, He 
known, we have 


a in terms 
41S a linear 
hee, when 71,18 


(8) 


" 
Mo + uy = 1 


* Elsewhere we have denoted this matrix Rp, but we are here using t he subscript ¢ for 
the primary vectors T: and reserving the subscripts р and q for the primary reference 
vectors A, В, and С. Hence the correlations of the primary factors are here denoted 
It, instead of Tis 


Gr 
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or 
(9) Te te = 1, 


so that the clement i4, is known. The other diagonal elements of U, are 
determined in the same way, so that, for example, 


(10) tho + tin = 1. 


When the matrix U: is known, we have, by (7), 


(11) Wag = Py 
or 
(12) Une = ADU, 


so that the transformation Vine is known. The saturations of tests j on the 
second-order general factor G and the group factors w can then be computed. 
The transformation matrix Жы represents а rigid rotation from one or- 
thogonal frame to another orthogonal frame, and hence this transformation 
matrix must be orthogonal by rows. A fourth row could be added to Vinw 
for a fourth orthogonal unit vector IV with cell entries which normalize 
each column. Then we should have an orthogonal matrix of order 4X4. 


Case 3. Transformation from У to U, including the group factors and column 
vector G 
Here it is assumed that the computations are to be made from the oblique 
factor matrix V. In Table 5 we have the transformation equation in rec- 


tangular notation, namely, 


(13) Vj pw = Uy; 


which gives the saturations of the tests j on the group factors and on the 


general factor. If the factor matrix V is extended to include the primary 
vectors Тү, we have the diagonal matrix D. Applying the same transforma- 


lion to D, we have 


(14) Dipl pwe = Us 


so that 


(15) Vow = DU. 
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When the elements of U, have been determined as for Case 2, the transfor- 
mation Yp, can be written by merely adjusting the rows of U: by the multi- 
pliers of Dz}. The transformation Wyo is then known, 

Table 5 


Case 3. Transformation from V to U, Including 
the Group Factors and the Column Vector G 


G 
x 


2 
x 
Ze 


ys 
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Case 4. Transformation from V to column vector G 
This is the simplest ease and perhaps the most use 
second-order domain. The matrix V js known in dete 


Structure and the primaries. The saturations of the te 


ful as regards the 
rmining the simple 
Sts J on the second- 


— — A 
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order general factor G are of interest, and these can be determined as linear 
combinations of the columns of V. Here we have the transformation shown 
in Table 5, namely, 


(16) Vio" as = fj 
Applying the same transformation to Dip, we have 
(17) Di Woo = Pigo 


The elements of the column vector r are known from the correlation ma- 
trix TT" = I, of the primaries. Then 


(18) Vig = Dit j 


and hence the column vector Yp is known. In computing, it is necessary 
only to multiply the elements ri by the corresponding diagonal elements of 
Dz} to determine Мру. The desired column vector, 7j,, ean then be deter- 
mined. 


A trapezoid population 

In previous studies of factorial theory it has been found useful to illus- 
trate the principles by means of a population of simple physical objects or 
geometrical figures. ‘The box population was used to illustrate three corre- 
lated factors and their physical interpretation. In the present case we want 
four factors in the first-order domain, which, by their correlations of unit 
rank, determine a general second-order factor. The correlations of three 

, H . 
Variables can nearly always be accounted for by a single factor, and hence it 
seems better to choose a four-dimensional system in which the existence of a 
second-order general factor is more clearly indicated by the unit rank of the 
correlations of four primary factors. For the present physical illustration 
we have chosen a population of trapezoids whose shapes are determined by 
four primary parameters or factors. 

‘The measurements on the trapezoids are indicated in Figure б. The base 
line is bisected, and the length of each half is denoted by the parameter c. 
An ordinate is erected at this mid-point, and its length is A. This ordinate 
divides the top section into two parts, which are denoted a and b as shown. 
These four parameters, а, b, с, and Л, completely determine the figure. The 
test battery was represented by sixteen measurements, which are drawn in 
the figure. The parameters а, b, c, and Л are given code numbers 1, 2, 3, 
and 4, respectively. Variables ( 12) and (13) are the two areas as shown, The 
sum of (12) and (13) equals the total area of the trapezoid. In general, each 
of these measurements is a function of two or three of the parameters but 
not of all four of them, and hence we should expect a simple structure in this 
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correlation matrix was factored by the group centroid method, and the re- 
sulting factor matrix F is shown in Table 8. The fourth-factor residuals are 
listed in Table 9, which indicates that the residuals are vanishingly small. 
Applying the rotational methods to the configuration, we found the trans- 
formation matrix A of Table 10, which produced the oblique factor matrix V 
of Table 11. In this matrix we are now concerned with all but the last col- 


Table 6 


Trapezoid Parameters 


a 
° 


1 
2 
Б] 
4 
5 
6 
7 
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12 
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umn. When pairs of columns of the factor matrix V are plotted, we have the 
configuration shown in the diagrams of Figure 6, in which a simple structure 
is clearly indicated. The cosine of the angle between the reference vectors is 
indicated on each diagram of Figure 6. These cosines were obtained from 
the relation C = A’A, as shown in Table 12. 

So far in the analysis we have found that four primary factors account for 
the correlations, and this corresponds to the fact that we used four param- 
eters in setting up the trapezoid figures. The four primary factors are cor- 
related, as indicated by the obliqueness of the reference axes in the dia- 
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grams of Figure 6. The next step is to determine the correlations between 
the primary factors that correspond to the primary reference axes. For this 
purpose the inverse of the matrix C is computed, as shown in Table 12. From 
the diagonal values of this matrix are found the numerical values of the 
diagonal matrix D, which is also shown in Table 12. The 
onal matrix is also listed. These numerical vs 
of the entries in D. 


inverse of this diag- 
lues are merely the reciprocals 


Table 8 


Table 9 
Orthogonal Factor Matriz F ышк ^f 
?esiduals 
1 
1 E 
2 759 
3 -79 
4 ‚81 
5 .88 
6 .87 
T .96 
8 .98 
9 ‚95 
10 .88 
П -78 
12 .97 
13 .98 
14 :21 
15 —.60 
16 —.46 


Table 10 


Table 11 
Transformation Matriz А Oblique Factor Matriz V 


A B e H A * 


I 07 .12 39 вз 
II 70 —.01 —.81 135 
III AL — 32 9B o4 
IV —301 -94 —.34 —.44 


"ex 


96 -1 
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In Table 13 we have the correlation matrix R,, showing the correlations 
between the primary factors. These are the cosines of the angles between 
the primary vectors. It can be seen by inspection that this matrix is close 
to unit rank, which indicates that a single general second-order factor can 
be postulated to account for the correlations between the primary factors. 
The saturation of each primary factor with this second-order general factor 
was determined by one of the special formulae for unit rank, and the satura- 


Table 12 


Matrix С = MA 


Matriz Dip 
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Table 13 


Correlation Matriz Ri = РС рь 


Column Vector rt 
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tions are listed in the column vector rj. The interpretation is, for example, 
that the primary factor A has a correlation of .71 with the second-order 
general factor G. The closeness of the correlation matrix to unit rank is 
shown by the small side correlations in the residual matrix of Table 13, The 
diagonal values of the residual matrix show that part of the total variance 
of each primary factor which it does not share with the ge 
factor. If the diagonals of this matrix vanished completely, then the pri- 
maries would have their total variance in common, and the original reduced 
correlation matrix for the tests would have been of unit rank. 

The saturation of each test with the second-order general factor was de- 
termined as a linear combination of the columns of the oblique factor matrix 
V of Table 11. The transformation of equation (18) was used, and the nu- 
merical values of Yp, were listed in Table 13. Column G of Table 11 was then 
computed by equation (16). 

The second-order general factor G сап be interpreted in this example as a 
size factor, and it also indicates that, in generating the thirty-two figures, 
the four parameters a, b, c, and h were not allowed to take entirely independ- 
ent values. In other words, either the extreme forms of figures did not occur, 
or else they were used only occasionally. If the four parameters had been 
allowed to take entirely independent values, then there would have been an 
appreciable number of figures in which one of these parameters had an un- 
usually small value, while some other parameter had some unusually large 
value. "This interpretation of the second-order general factor leads to a con- 
sideration of what we shall call generating parameters. The present geo- 
metrical example illustrates the type of factorial organization that is repre- 
sented diagrammatically in Figure 2. The problem of interpreting the four 
primary factors can be solved in this case without investigating the second- 
order domain. But if the correlations between the primary factors show un- 
expectedly low rank, then this fact can be utilized factorially in g 
ther insight into the conditions under which the objects wı 
The four primary factors here identified by the simple struc 
four parameters that were used in setting up the problem, 


neral second-order 


aining fur- 
ere generated. 
‘ture were the 


Generating parameters 
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simplifying set of parameters representing caus- 
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ative factors, the factorial methods become even more useful as tools in sci- 
entific work. 

The numerical values of the trapezoid parameters in Table 6 defined 
thirty-two figures of various shapes. The method of constructing the table 
of four measures for each figure determined whether one or more second- 
order factors would be present and also whether each of the primaries would 
be equally or differently represented in the second-order factor. The fac- 
torial result could be altered indefinitely by the manner in which the objects 
were generated in constructing Table 6. Since it is the object of factor anal- 
ysis to reveal the underlying order in the domain, it is an essential part of 
the numerical example to show that there is a relation between the generat- 
ing principles and the factorial results. 

The first column of the table contains the three linear measurements 1, 
2, and 3. Suppose that these were inserted in the column entirely at random. 
Assume that each column was similarly constructed by distributing a set 
of measurements entirely at random. Then we should expect zero correla- 
tions between the four primaries, Ta, Tg, Tc, and Ty. The correlation ma- 
trix for the four primary factors would be an identity matrix, and it would 
not be factored because the primary factors would be statistically independ- 
ent. There would be no second-order factor present. 

If, for each one of these thirty-two figures with uncorrelated primaries, 
we should draw another one similar in proportions but with twice the area 
and another one with similar proportions but three times the area, then we 
should have a set of ninety-six figures, consisting of three sets that have 
similar shapes but different sizes. If this new set of ninety-six figures were 
analyzed factorially with the same battery of sixteen measurements, we 
should find the same primary factors, but they would be correlated. Fur- 
thermore, the correlations of the primary factors would all be the same, so 
that we should have a correlation matrix for the primary factors with uni- 
form side correlations. The reduced correlation matrix would have unit 
rank, and all the four primaries would have the same saturation on the sec- 
ond-order general factor. This would be a situation with a second-order 
general factor which has a uniform effect on all the primaries. Here, again, 
the factorial result would be determined by the manner in which the objects 


were generated. 
Suppose that a group of persons was asked to draw some trapezoids of 


arbitrary shapes and that these trapezoids were assembled as a population 
of figures to be measured and analyzed factorially. Then we should almost 
certainly introduce a second-order size factor because our subjects would 
probably unwittingly draw the figures so that the several dimensions of each 
figure would be at least roughly of the same general order of magnitude. 
Some of the subjects might draw trapezoids of the general size, of say, 5 or 
6 inches, while other subjects might draw figures only 1 or 2 inches across 
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Very few would produce trapezoids that are 1 or 2 inches wide and 10 pied 
tall. In other words, since some subjects would draw big figures and others 
small ones and since they would probably produce very 
there would be strong correlation between the primary factors, and these, 
in turn, could be analyzed factorially into secondary factors. In this situa- 
tion the rank of the correlations of the primary factors would probably not 
be exactly 1, but the inference could certainly be drawn from the factorial 
result that secondary factors were operativi 


e to produce some big figures and 
some small ones in addition to the primary parameters that define the in- 
dividual figures. 


The interpretation of the second-order factor as a size factor in the trape- 
zoid example should be distinguished from the size factor that could be 
chosen as a parameter in the first-order domain. If one of the measurements 
had been the total area of the trapezoid, it would have been represented by 
a test vector in the middle of the configuration, since it would be affected by 


all four of the generating parameters that were used and which 
the simple structure, The total-area 


unit vector, and it could be used 
trapezoids. It would not be iden 
they would be closely re] 
or a second-order facto 
the figures or objects 


few extreme figures, 


appears as a first-order 
r depends on the restrictive conditions under which 
d or selected and also on the selection of 
attery. It is interesting to note that here the 
that the thirty-two trapezoids had been sys- 
restrictive conditions or else that the objects 
ed under some restrictive conditions. 

are clear in both the first-order 


and the second- 
àn Sometimes be drawn concerning the generat- 
ed the individu 


r al parts of the objects. Such in- 
ferences can be the basis for potheses that can be investigated 


more directly controlled experi- 
ments. 


Incidental parameters 

So far we have considered the primary f. 
Structure as representing parameters that can be given some scientific inter- 
pretation in terms of concepts that are fundamental for the domain in ques- 
tion. In using the simple-structure solution, which leads sometimes to the 
second-order domain, we have tried to avoid using arbitrary parameters 
whose only merit is that they serve in the condensation of numerical data. 
We have tried to find in the primary factors a set of parameters which not 
only describe the individual measurements but also reveal something about 
the underlying order in the domain. In looking for meaningful parameters 


actors determined by a simple 
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of this kind it would be an error to assume that all the factors have signifi- 
cance that transcends the particular experiment or the particular group of 
subjects. It would be strange indeed if factor analysis were immune from 
the distracting circumstances of the particular occasion. The experimenter 
must try to distinguish that which is invariant and which transcends the par- 
ticular experimental arrangement or the particular experimental group of 
subjects from that which is local and incidental to the particular occasion. 
In factor analysis we are not relieved of this difficult. task any more than 
in other forms of scientific experimentation. In order to focus attention on 
this circumstance, it might be well to distinguish the primary factors which 
represent the invariants for which we are really looking from those primary 
factors which, though genuine as regards the explanation of the test vari- 
ances, are local and of significance only for the experimental group or the 
particular occasion. Primary factors which characterize only а particular ex- 
perimental group or a particular situation may be called incidental factors to 
invariants which are normally the object of scientific 


distinguish them from the i 4 
appear in the first-order or in the 


experimentation. Incidental factors may 


second-order domain. ; Ку: 
А few examples will serve to illustrate the manner in which incidental 


ar as primaries in factorial analysis. In addition to the 
e found in different groups of subjects, we 
are unique for the particular occasion. Sup- 
pose that an exceptionally good examiner who is skilled in obtaining good 
rapport with the subjects should give a part of the test battery to a part of 
the experimental group. A primary factor might appear for this group of 
tests, and the investigator might be at a loss to explain it because he would 
be thinking about the nature of the tests and he would try to find something 
Common in the psychological nature of these tests. It might not sli to 
him that this is the very group of tests that was administered by the experi- 
i tor would probably be left without interpreta- 
lion in the final results, or the interpretation might be one that we ау 
be sustained in a subsequent experiment with йге subjects and ла 
ent examiners. Incidental factors are almost certainly pea F е: 
Study. Hence the investigator should feel free to Mrs em: e in E a- 
tion those primary factors which do not lend therse m eg ies я sci- 
entific interpretation. Even then the шаш ү ae Au А їй 
the nature of a hypothesis to be sustained, if pe sj ету et 
torial studies. The fact that all the variances Ж E i nein E п | 
for in the interpretation has led some students се E Lx bra 
result should be disearded, but that is not ies ea y un ud 0 
Make an important discovery concerning ШЕ PIR : i- 
еп though the majo! part of the common-factor 
nt, Seed It is assumed, of course, that such a finding 


factors may appe 
primary factors that would b 
might find primary factors that 


enced examiner. Such a fac 


five in an experime 
Variances remains unexpl 
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could be sustained by the construction of new tests with prediction as to 
how they should behave factorially in new groups of differently selected 
subjects. 

In one factorial study it was found that a primary factor was common 
to a set of tests that were given by the projector method with individual 
timing for each response. The interpretation of such a factor was uncertain. 
Some psychological function might be involved in the projector tests which 
was absent from the other tests; but the explanation might also be that some 
motivational condition was common to the projector tests that was 
from the other tests and which would be of only incident 
far as the major purposes were concerned. 

Suppose that one of the examiners misunderstands the time limits for a 
set of tests and that he gives the shorter time limits to a part of the group of 
subjects for some of the tests. A factor might appear under certain cireum- 
stances that would be incidental and of no fundamental significance, but the 
primary factors that are significant might still be revealed. An unexpected 

interruption in a school examination, such as fire drill, a street parade, or 
the anticipation of an important school event, may act to introduce inci- 
dental factors, 

One of the most important sources of incidental factors is to be found in 
the selective conditions, If a group of subjects is selected because of quali- 
fication In à composite of two or more tests, the unique variances of such 
selective tests combine to form one or more incidental common factors 
Which would have remained a part of the unique variance if the selective 
conditions had not been imposed. The correlations between the factors are 
determined in large part by the selective conditions. If a group of subjects 
is selected because of certain test qualifications, it is to be expected that the 
primary factors will show correlations between factors that are different 
from the correlations between the same factors in an unselected population. 
It must not be assumed that the factors are different just because they cor- 
relate differently in different Populations. This effect is well known with 
physical measurements—height and weight with intelligence, for example— 
whose intercorrelations are determined in large part by the selective condi- 
tions. These changes do not affect the identity of the factors. An incidental 
factor which is introduced by conditions of selection may be trivial or it may 
be significant, depending on the nature of the unique variances which are 
introduced into the common factors by the selective conditions. 

It should be remarked that in a well-planned factorial experiment the 
incidental factors are usually of secondary importance in comparison with 
the variance that is assignable to the principal primary factors for which an 
experiment was planned. When one or more primary factors have relatively 
small variance and do not Seem to lend themselves to clear interpretation, 
they should be reported without interpretation, Some reader of such a re- 


absent 
al significance as 
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port may find a fruitful hypothesis for it, or the factor may be of only in- 
cidental significance. 

These few examples will serve to call attention to the fact that not all the 
primary factors can be expected to have meaning in the fundamental sense 
of representing functional unities whose identity transcends the particular 
occasion. It does not follow that incidental factors are in any sense arti- 
facts. They may represent genuine factors that were operating to produce 
the observed individual differences, but their significance may not extend 
beyond the particular occasion. In that sense they are irrelevant to the 
purposes of the experimenter, even though they are valid as factors which 
can sometimes be identified. ; 

An interesting application of second-order factors is an attempt {о recon- 
cile three theories of intelligence, namely, Spearman's theory of a general 
intellective factor; Godfrey Thomson’s sampling theory, with what he 
calls "sub-pools"; and our own theory of correlated multiple factors, 
which are interpreted as distinguishable cognitive functions. The tetrad dif- 
ferences vanish when there are no primary factors common to the four tests 
of each tetrad, the correlations being determined only by the general second- 


order factor. 


CHAPTER XIX 
THE EFFECTS OF SELECTION 
Types of selection 


Factorial results are affected by selection of subjects and by selection of 
tests. It will be shown that the addition of one or more tests which are 
linear combinations of tests already in a battery causes the addition of one 
or more incidental factors, If the given test battery reveals a simple struc- 
ture, the addition of tests which are linear combinations of the given tests 
leaves the structure unaffected, unless the number of incidental factors is 
so large that the common factors become indeterminate. Such a situation 
violates the restrictive relation between the number of tests л and the num- 
ber of common factors r. 

When a factor analysis has been completed for a test battery on one group 
of subjects, the administration of the same battery to a differently selected 
group of subjects will give a different factorial result if the second group is se- 
lected by any criterion in the common factors. If a simple structure is 
found for the first group of subjects, the sa 


me structure will be found for the 
second group, but the correlations between the primary factors will be dif- 


ferent. Incidental factors are added to the common factors under certain 
conditions of selection of the subjects. It will be shown that, under these 
various conditions of selection of tests and subjects, a simple structure re- 
mains invariant. It will also be shown that the physical interpretation of 
each primary factor remains the same for wide variations in selection of 
both subjects and tests but that their intercorrelations are altered by selec- 
tion of subjects. It ean readily be seen that these considerations are of 
fundamental significance for factor analysis as a Scientific method, 


These principles will be illustrated by numerical examples with the box 
problem. The box dimensions, 7; y, and z, may have low correlations in one 
group of boxes, but these correlations may 
is repeated on another group of boxes so во] 
tively low and squatty. Then the z-variance is reduced, and the c. 
may be altered. Of course, these altered correlations of the prim 
do not deny the identity of the parameters, length, height, and width, for 
describing the box measurements in both box populations, The factors are 
the same and their physical interpretations are the same in both popula- 
tions, even though their intercorrelations alter from one experimental popu- 


lation to another, This simple Principle will be illustrated in numerical ex- 
amples. 


be quite different if the 


analysis 
ected that they are all, s 


ay, rela- 
orrelations 
ary factors 
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Experimental dependence 

One of the most important sources of incidental factors which are extrane- 
ous to the purposes of a factorial study is what we have called experimental 
dependence. In earlier studies with the factorial methods this source of fac- 
tors was not recognized, but for a number of years this principle has been 
applied in the preparation of test batteries in the Psychometric Laboratory 
at the University of Chicago.* In Table 1 we have the factor pattern for a 
set of five tests (1, 2, 3, 4, and 5) with two common factors, A and B. The 
reduced correlation matrix of this battery would be of rank 2, corresponding 
to the number of common factors. For each of these five tests there is also 
indicated in the factor pattern a unique variance. Now suppose that the 
performance from which the score of test 1 was derived is also used to derive 


Table 1 

A B Unique Factors 
1 x x х 
2 x x x 
3 x x x 
4 х х x 
5 x x x 
la х x x 


another score, la. Then we have two test scores derived from the same per- 
formance, and they are quite likely to share some of the uniqueness of that 
particular performance. This fact is indicated in the factor pattern by the 
two x's in the third column. Tests 1 and Та share the same uniqueness. In 
addition, the two tests 1 and 1a may also have unique factors that are not 
shared, but these are not indicated in the factor pattern of T'able 1. 

A simple example of this manner of constructing a test battery would be 
to record speed and accuracy as two scores from the same test performance 
in the attempt to cover all aspects of the domain that is being investigated. 
The new factor pattern, after adding test la, has three common factors. 
The third common factor has saturations in only 1 and 1а. It should be 
noted that the rank of the reduced correlation matrix has been raised from 
2 to 3 by this manner of assembling the test battery. In factoring the test 
battery it would be necessary to extract one additional factor from the cor- 
relation matrix before the residuals vanish. After rotation of axes, a third 
common factor would be found on which only two tests have significant 
saturations, namely, 1 and la. Upon examination of their nature it would be 
found that these two tests derive from the same test performance. Such a 


factor on which two variant scores of the same performance have satura- 


* The importance of experimental dependence in factor analysis was first pointed out 


by Ledyard Tucker. 
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tions is called a doublet factor, and it is ignored in interpretation. It rarely 
gives any leverage on the problem of interpreting what the factor might 
represent, except that it is something concerned with that test performance 
—a fact that was known before the factoring was begun. For this reason it 
is best to avoid inserting in a test battery two or more measures that are 
taken from the same test performance. 

If we push this effect to an extreme by adding a duplicate form of each 
test to the battery, or by using each test and a paralle 
have a battery of ten tests instead of the five tests with which we started. 
If there is a unique variance for each test, then the factor pattern of Table 1 
is altered by introducing five new common factors so that there üre seven 
common factors in all, the two factors common to the five original tests and 
the five unique variances which have now become common factors. It is 
doubtful whether a useful resolution of the correlation matrix into common 
factors is then possible. The unique variances have been m 


ade into common 
variances by the inclusion of a parallel form of each test, and the common 


factors that might be significant would be obscured by the addition of five 
common factors that, may make the factor problem insoluble. 

It is good practice to inspect a test battery for experimental dependence 
before factoring is begun. One of the most common forms of experimental 
dependence is that in Which some kind of total score is added to a test bat- 
tery to be factored. A mental age, for example, is essentially a summation 
Score. It is known before the analysis is begun that the total Score will be 
Tepresented by a test vector that is in the middle of the configuration and 
that hence it contributes nothing to the identification of the primary vec- 
tors which are at the corners or intersections of the configuration. Nor does 
it contribute to the identification of à second-order general faetor which is 
defined by the correlations of the primaries, However, a total score may be 
found to have higher saturation on the second-order general factor than on 
any one of the primaries. The best procedure is to investigate the structure 
of the domain without the summation score and to investigate the char- 
acteristics of the summation Separately. 


l form of it, then we 


Linearly dependent tests 


The summation score is an example of both experimental dependence and 
linear dependence of scores, If we add to a test battery a new test score 
which is merely some linear combination of test scores from the battery, we 
have linear dependence which generally introduces a new common factor, 
An exception is that in which the tests that are combined have no unique 
variance—a situation that arises only rarely. The introduction of a new 
test which is a linear combination of tests already in the battery is represent- 
ed by the factor pattern of Table 2. Here we re 


epresent five tests (1,2, 3, 4, 
and 5) with two common factors and five unique factors, If a new test were 
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added by merely taking the sum of the test scores 1 and 2, we should have 
the test denoted (1+2). This test is indicated as participating in the two 
common factors A and B and in the uniqueness which adds two more com- 
mon factors. 


Univariate and multivariate selection of subjects 

When a factor study has been done on a particular group of subjects 
with a given test battery, it is of fundamental interest to know how the 
factorial results would be altered if the analysis were repeated on another 
group of subjects differently selected. Here we are not interested in whether 
one of the groups is in any way representative of a universe while the other 
group is not so representative. For our purposes the two groups are co- 
ordinate, but they are differently selected so that they differ in homogeneity 
in one or more factors. The results for either group are assumed to be 
known, and we want to be able to predict the factorial results when we know 
how the second group was selected in comparison with the first group. 


Table 2 
1 B 1 2 3 4 5 

1 x х x 

2 ж. A х 

Б x xX х 

4 X x х 

5 х x х 
о ж 


Consider the selection of a second group of subjects so that they are more 
homogeneous, or less, in one of the tests in the battery. This test will be 
called the selection test, and it will be denoted 7. The variances of all the tests 
arbitrarily reduced to unity by standardizing the scores for the 
first group so that in that group о; =1 and, in particular, so that a= 1. Let 
the new group be selected so as to have a different standard deviation in the 
selection test J, and let it be denoted s;z£1. The new group of subjects will 
have different standard deviations in all the tests that have correlation with 
the selection test J. These new standard deviations will be denoted s;. The 
problem then is to estimate the new test correlations and the new factorial 
results for the second group. The selection of a new group on the basis of 
their homogeneity in one of the tests has been called univariate selection by 
Godfrey ‘Thomson, who has investigated this problem and whose results 
will be summarized here. Univariate selection can be either complete or 
partial. 1f the new group of subjects is so selected that all the subjects have 
the same score in the selection test l (c; = 0), then we have complete univariate 
selection. If the new group is selected so that 9170, then the selection will 
be called partial univariate selection as to the selection variable 1. It will be 


have been 
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shown that interesting differences in the factorial results are determined by 
univariate selection when сг> 0 and when 7:—0, which represents complete 
homogeneity in the selection test. Further, it will be shown that the fac- 
torial results are also affected by the communality of the selection test /, so 
that the results differ according to whether h?=1 or hil. 

When the selection of a second group of experimental subjects is deter- 
mined by more than one selection test, we have multivariate selection. The 
principles of multivariate selection can be illustrated in terms of two selec- 
tion tests, denoted I and m. Such a situation can be analyzed as two suc- 
cessive univariate selections. In multivariate selection one is likely to be deal- 
ing with a composite selection test, c=f(l,m). This is the case of multivari- 
ate selection that has been studied by Godfrey Thomson and by Walter 


x E 


Ficure 1 


Ledermann. Here, also, the factorial results are affected by complete and 


partial selection in the composite criterion of selection and by the comm unal- 
ities of the selection variables. The theory of ашуг 


) 4 : ariate and multivariate 
selection will be described and numerically illustrated, and 


] à comparison 
will be made of the psychological interpretations of the гези 


ilting factors, 
Geometrical representation 


1 anga ) ste ations, we 
shall consider the geometrical interprets ion in terms of the test 
configuration. In Figure 1 we ectors in a eon 
: : : E " ^ d " 
figuration of two dimensions, m will be simplified 
by letting all six of these test vi 


nality in the space of 
the two common factors, so that the diagram represents the total test space 
‘tions of the tests are represented by the unit 
length of the test vectors, which corresponds to the fact that o;=1 for all the 
tests. The test correlations are represented by the scalar products of pairs 


of test vectors. In n dimensions we should have a test configuration which 
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is spherical, in that the dispersions have all been normalized by reducing the 
scores to standard scores. The correlations are unaffected by this standard- 
ization. 

Now let another group of experimental subjects be selected so that they 
show less variability in the direction of a selection test, L. The effect will be 
to alter the shape of the test configuration, so that it is narrower in the direc- 
tion of the selection test. In fact, the configuration will have an ellipsoidal 
shape, so that the configuration of two dimensions in Figure 1 will be an 
ellipse determined by the termini of the test vectors. Since the new group is 
selected so as to be more homogeneous in test L, it follows that the compo- 
nents in this direction of all the other tests will be affected proportionally. 
If the factor of test L is reduced in dispersion, the same factor is also reduced 
proportionally in all the other tests in which this factor enters as a compo- 
nent. 

Figure 1 has been drawn to represent a reduction of one-half in the dis- 


persion of test L. The L-component of the other tests is then also reduced 
by one-half, so that the new test vectors are depressed in the direction paral- 
lel to test Г, as shown in the diagram. The termini of the new test vectors 
now define an ellipse instead of the circle. It should be noted that tests A 
and E are unaffected by the selection because these tests are orthogonal 
(uncorrelated) to the selection test. 

The scalar product of the unit vectors L and D is the given correlation 
rip. This correlation is also the cosine of the angle LOD, and it is the pro- 
jection of the vector D on the vector L. This is the distance DX. It is the 
the direction of L. Reducing this component by one- 
half, we have the new test vector d as shown. The length of this vector is 
the standard deviation of test D in the selected group. The new vector for 
test L is 1, which is collinear with L but only half as long. The cross product 
ld is the covariance of tests L and D in the new group. If each of these short- 
ened vectors, l and d, is extended to unit length, we have the vectors L' 
and D’. It should be noted that the vectors L and L’ are identical, whereas 
D and D' are not the same vectors. The correlation between two tests, such 
as C and D, is changed by the selection to a value which is the scalar prod- 


component of D in 


/ 
uct of the new vectors C” and D'. 


The correlations after selection 

ariate selection on the test correlations can be shown 
veetorially. In Figure 2 let L be a unit vector representing the selection 
test in the total test space. Let J be a unit vector representing any other 
test in the battery. The components of J may be considered to be OX, 
which is orthogonal to L, and XJ, which is collinear with L. When selection 
is made on the basis of dispersion in L, the component OX of J will be un- 
affected, whereas the component XJ will be affected. 


The effect of univ 
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Let the selection alter the dispersion of L 
is altered from ¢,=1 to &= p. The test L 
vector l, whose length is s; =p. The new te 


so that its standard deviation 
is then represented by the test 
St vector Г can be written in the 


form 

(1) l=L-¢th, 
where 

(2) t=1-p, 


The component XJ of J is the projection of J on L, and it is also the corre- 


4 


1 


1 


x 


Ticurr 2 


lation rj. Since the effect of selection on the te 
its component collinear with L, we have 


(3) 


st vector J is Proportional to 


Jo=J— tral, 


where Jo is the new y 


ector for test J. Its new st 
length of the vector Jo 


andard deviation is the 
. Squaring (3), we have 


(4) Jo = (7 = tral) 
or 
(5) R= и 


= 0° — Qtr + eua. 


The sealar product JLis the corre] 
Hence 


(6) 


ation rj and the scalar product [2 — J2— i 


8 = 1 — 203, + Cr}, 
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which becomes 


(7) s$21—r75Q-t). 
For convenience let @=1—p. From (2) we then have 
(8) ж —{Є=@, 

so that the new variance s? сап be written as 

(9) ў =1- 0%, 


by which the new standard deviation becomes 


(10) 


The numerical values of s; can then be computed, since the correlations rj; 
and the constants р and ф are assumed to be known. 
By analogy with (3) we сап write the corresponding vector equation for 


test K, and we then have 

(11) Ko = K — trial. 
The new covariance cj; can be expressed as the scalar product JoKo, which 
by (3) and (11) is 


(12) ej = JoKo = (J — triL)(K — trul) ; 


2 


and this becomes 

(13) ej = ЈК — traJ L — trik + Prr. 

Recalling that the scalar products JK = rjj JL=rj, КІ, = га, and L?=1, 
we get 


(14) cg = rji = tia — ты + Crair. 


Collecting terms, we obtain 


(15) cj = т — rara(2t — Ё), 


which, by (8), becomes 

(16) сук = Tik — Фут 
This equation enables us to compute the new covariances to be expected by 
selection on test L. 
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from which we have 


Е 1—7 —À 
(28) Tu = 


which is the Otis-Kelley formula. These estimates of the effects of selection 
are, of course, in the nature of expected averages, from which the selected 
samples would show random deviations, 


The communalities 


So far we have considered the effect of selection on the dispersions and 
correlations which imply the total test space. When the communalities are 
known in one correlation matrix, the new communalities can be found by a 
slight adaptation of equation (18). The diagonal correlations are then writ- 
ten rj;=h? instead of unity. Since the selection is assumed to be 
the test scores, it follows that the uniqueness of the selection test, as well 
as its common-factor variance, must be involved in the result, 

Writing equation (18) for the communalities 


made by 


› Where 755 =, we have 


(29) 


» _ А — Фф, ; 
Sj = h} = 1, G #1), 
where the denominator is written by equation (10). It should be noted that 
equation (29) applies to the computation of all the communalities, except for 
the selection test. Hence the restriction for equation (29) that j=l. 

When j=l, we can write this equation in the modified form 


(30) n= it = Pe 


Pi Б? 
ати = hp = ET EE “М - 
1 — eruh? 
where one of the correl 


ations ry, is unity and the other is hi. Then 


р 2 Ra = 7) hn 
31) vp ШИ Ф) ир 
( 1 e qh? jm Ph? ’ 


by which the new communality 
tion (31) may be derived vectorially as follows: 

Let L in Figure 3 be the selection test vee 
let Le and Ly be the common-factor compon 
of L, so that L=L.4+Ly. By selection the configuration of test vectors is 
first contracted by the proportion p in the direction of L and then normal- 
ized. Contracting L to pL and then normali 


i p zing reproduces L again. Hence 
the selection vector L is not altered by selection, except, of course, when 


of the selection test can be computed, Equa- 


tor in the total test space, and 
ent and the unique component 
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p=0, in which case pl becomes а null vector and the configuration is reduced 
to dimensionality (r— D. 

The length of the common factor component Le is hı, and the projection 
of Le on L is pL, whose length is hj. The correction vector for L., which is 
introduced by selection, is therefore — th? L, as shown in Figure 3. The al- 
tered vector Le ean now be written as 


(32) dL tb, 


where „/ represents the distortion of Le that is introduced by compressing 


the configuration in the direction of L before the vectors are finally normal- 


ized again. 


FIGURE 3 


In order to find the length of „LL, we write 


GL)? = Lè 1, + PhiL. 


(33) 

Since L2=1, L2 - hi; and, since the scalar product L,L — Л, equation (33) 
becomes 

(34) GL)? = М — ЖМ + Phi, 


which, by (8), reduces to 


(35) (Li? = M FB: 
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А in Figure 3 с: rritten either as a 

The scalar product of »L; and L in Figure З can be written eithe P 

product from equation (32) or as the product of the lengths of L anc PPM 

and the cosine of their angular separation. The length of L is unity, the 

length of ,L/ is hı V1— qh? ; and, since the cosine of the angular separation 

is also the projection of L on the new common-factor space, namely, „hi, we 
have 


(36) Iob = hi + hiv] — qhi. 
The same sealar product can be written by equation (32) 
(37) «= Ар. hil? , 


which becomes 


(38) LIL = ht — thè 


or 
(39) 


Fquating (36) and (39), we get 


(40) hip = Љут qi; 
and writing this explicitly 


for the new communality of the 
we have 


selection lest, 
(31) sh 


as previously written, By this equ 
tion test can be computed, 

In solving a complete problem of this kind it is cc 
frey Thomson's equation (18) for 
communalities, except that of the 
applies, 


ation the new communality of the selec- 


venient to use God- 
all the new correlations, including the 


Selection test, for which equation (31) 


Computational Sequence 


The problem of determining the effects of univariate selection starts with 
the correlation matrix and the communalities, as well as the degree of selec- 
tion on the selection test, which is denoted /, with subscript 1. It is assumed 
that the given scores have been standardized so that с;= 1 for all tests. The 
selection of a new group is such that the standard deviation of the selection 
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test is changed from the given value с; = 1 to a new value, sı, which is known. 
The ratio of the standard deviations is 


where p is a parameter that determines the new factorial results. The fol- 
lowing convenient constants are then computed: p°, q = 1—р°, and q. 

The given correlation matrix is written with unity in the diagonals. 
Equation (18) ean be written in the form 


(41) В, = МСМ = M(R — UU')M , 


where /?, denotes the expected correlation matrix after univariate selection. 
The matrix U is a column vector with elements и; = гу, and the matrix M 


Table 3 


Given Correlation Matriz Ry 


: З s 4 6 7 8 9 10 

1 | 1.000 .000 .000 .606 .350  .606 .000  .000 

2 .000 1.000  .000 .350 .000  .000  .350  .606 

3 "000 000 1.000 ..000 .000  .606  .350 .606 . 

4 “3590 606 .000 1.000  .606  .175  .303 .303 . 

5 боб .350  .000  .606 1.000  .303 .525  .175 .303 

6 7350  .000  .606  .175 03 1.000 ..606  .525  .303 

7 606.000  .350 .606 1.000  .303  .175 

8 .000  .350  .606 5  .303 1.000  .606 

9 .000  .606  .350 3 “606 1.000 
10 “404 .404 404 552 '552 — 1552 


is a diagonal matrix with elements d;—1/s;. Equation (41) indicates the 
computational order. One may compute, first, the column vector U, then 
the covariance matrix C= UU’ of order nXn, and, finally, R.. The correla- 
symmetric, and it also has unity in the diag- 


tion matrix 2, is, of course, 
onals. 
When the correlation matrix R, has been computed, the communalities 


may be computed separately by equation (36), where it must be remembered 


that ти is written as Л. 


Examples of univariate selection 
ples of the preceding sections will be illustrated by a numerical 


The princi 
A variant of the box problem will be used. 


example with a physical model. 
The factor matrix of Table 4 represents ten measures of the box problem, in- 
cluding the basic parameters, height, width, and length, and such other box 


measurements as the diagonal of each face, the area of each face, and one 
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complex measure, such as the volume. In order to illustrate the effects of 
selection, the given example in Table 4 is arranged to represent the situation 
in which the three basic parameters are uncorrelated. Further, in order to 
show the effect of selection on the communalities, these have been 
in the factor matrix as markedly below unity 
ures. The corresponding correlation m 
the given data. 


arranged 
‚ namely, .70 for all ten meas 
atrix is shown in Table 3. These are 


Table 4 


Given Factor Matrix Fy 


I II II 

1 .546 
2 
3 
4 
5 
6 
T 
8 
9 

10 

T'able 5 
Computation of 5] 
ти ти Uu s 


i sj т 


1.000 


-800 .640 360 -600 1.667 

000 .000 .000 4 -000 1.000 1 -000 

| «000 .000 1 000 1.000 1.000 

| 280 .078 доо» .960 1,042 
.485 .235 765 


949 1.144 
.922 060 | .042 
765  .875 | -144 
1.000 
1.000 1.000 1.000 
-896 945 | -057 


. 280 
.606 485 
000 000 
000.000 
-404 .32; 


5 


It will now be supposed th 
is more homogeneous in the r 
viation is .60 in the new 


at a new collection of boxes i 
neasurement of test 1, so th 
population instead of unity, 
lation. The problem is now to investigate the effects o 
correlations and on the simple Structure, The reduc 
is the parameter p of Table 5, and from this value 
parameters, such as q, t, and the new standard ‹ 
measures that are correlated with test 1, These аг 

In the same table we have the detaile 


S measured which 
at its standard de- 
as in the given popu- 
f such selection on the 
ed standard deviation 
are determined the other 
leviations sj of the other 
e all listed in Table 5. 


d computations for the new standard 
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deviations s; and the reciprocal m;. The computations of this table are de- 
termined by equations (9) and (10). 

The new covariances after selection are determined by equation (16) 
which can also be written in matrix form, as shown in Table 6. The niies 
in this table are the cross products of the new scores, whose standard devia- 
e these entries are not correlation coeffi- 


tions have been depressed. Hene 
ariances of the tests. The corre- 


cients. The diagonal entries are the new v 
Table 6 


Covariance Matriz Cs = R, = UU’ or cs = гук — ujük 


1 ә 3 4 5 6 7 8 9° 10 
1 .360  .000 ..000 1%. .126 “146 
2 "O00 1.000  .000  .606 .350 .000 “404 
3 ‘Doo 000 1.000 .000 .000 606 7404 
1 126  .606  .000  .922 7 1462 
5 216 300 000 . `395 
6 126 000 .606 .097 “462 
7 ‘a18 000 .350 .167 1395 
8 000  .350 -606 : 
9 1000.606 .350 
10 146 AOE 104 


Table 7 


New Correlation Matriz Rs = МСМ 


1 š 3 4 5 6 7 8 á 1 
i 1006 000 000 -219 416.219 000 000  .256 
2 "000 1.000.000 632 1400.000 350 .606 .427 
3 ‘ooo. 000 1.000 000 000 2 Об 30 427 
4 “B19  .632 .000 1.000 1560.105 316 .547 .508 
5 "6  .400 -000 .560 1. .199 200 2817 48 
6 “319 .000 =632 -105 1.000. .547  .316 .508 
T 7316. „000 „100 198 560 1.000  .347 .200 178 
& | oo .350 «006 -310 547 347 1.000 606 -583 
9 | “ош 606 -350 -34 ‘310 2200 006 1000 -583 
10 | ‘956 427 427 -508 “508 0478  .583 .583 1.000 


sponding correlation coefficients are shown in Table 7, and these represent 
Godfrey Thomson’s equation (18) with unit diagonals. They can be ob- 
` { Table 6 by the stretching factors m;=1/s;, 


tained from the covariances 0 і ds ors? 
which are the elements of the diagonal matrix M. This matrix is identical 
with the table of covariances; except for the fact that the standard devia- 


tions have been stretched to unity. А . 
The new communalities ате determined by the detailed calculations 
shown in Table & from (36), in which ry is written as hı. Finally, when the 


new correlation matrix with communalities in the diagonal cells is factored, 
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we get the new factor matrix F,, as shown in Table 9. Here the number of 
factors is again three, as in the given correlation matrix, thus verifying God- 
frey Thomson’s theorem that the rank of the correlation matrix with com- 
munalities remains unaltered with univariate selection. However, 
ception should be mentioned. In complete or total univari 
factor or its equivalent—a test with perfect communality—-the new stand- 
ard deviation of the selection test or factor is zero, and the rank of the new 
correlation matrix has then been reduced by 1. In the present example the 
rank of the new correlation matrix, after complete selection by a factor or 
by a perfect test, is reduced to 2. Several examples of this type will also be 
shown. 

Perhaps the most important consideration in t 
what happens to the identification of the prim 


an ex- 
ate selection on a 


his problem is to ascertain 
ary factors. In Figure 4 we 


Table § 


| $.— В ari Table 9 
New Communalities: shj = ل‎ HT . я 
1 = qr New Factor Matriz Fs 
h rit rh VU Num. Denom. КН І II III 
1| .700 .837 :700 .448 252 -552 .457 1 —.340 . 688 
2 | .700 .000 .000 .000 .700 1.000 . 700 2 -048 
З | .700 -000 .000 .000 .700 1.000 - 700 3 —.474 
4| .700 .196 ‚016.010 .690 .990 .697 4 55 
5 | .700 .218 ‚046.031 1670 .970 .691 5 
6.700 .126 016.010 . 690 .990 . 697 6 
7| .700 .218 -048 .031 .670 .970 .691 7 
8 | .700 -000 .000 -000 .700 1.000 - 700 8 
9| .700 .000 .000 .000 -700 1.000 . 700 9 
10 | .700 .146 021 .014 686 . 986 .696 I( 


have the configuration of test vec 
In this diagram the test vectors ai 
vectors, so that the three-dimensional configuration ean bo Seen in a plane. 
The given positions of the test vectors are denoted а, and the new positions 
by b. A glance at this figure shows th 


at the position of the selection 
able 1 remains unaltered. So do also the tests which are uncorrelated with 
the selection test. These are tests 2, 3 


3, 8, and 9. These remain unaltered. 
The other test vectors—4, 5, 6, 7, and 10—moye in a direction radially 
from the selection vector, so that the triangular configuration js still re- 
tained. In the present example the actual dimensions of the triang 
remain unaltered. We see, then, that in this example of parti 
selection the simple structure remains invariant. If two inve 
ed with the two correlation matrices, one with the 
the other with the specially selected group of obje 
the same simple Structure, and they would identi 


tors before and after selection on test 1. 
ʻe represented by the method of extended 


rari- 


le even 
al univariate 
stigators start- 
first group of objects and 
cts, they would arrive at 
fy the same primary fac- 


== p-— 
RR 
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tors or parameters. But their results would differ as to the correlations be- 
tween the measurements with which they started. In the present example 
the correlations bet ween the primary factors would remain the same for the 
two groups of objects. Variations of these principles will be shown in addi- 
tional examples. 


FIGURE 4 


Phe most important principle to be drawn from analyzing the effects of 
selection on factorial results is that under wide variations in the conditions 
of selection the simple structure is invariant, so that the primary factors or 
parameters are the same. The correlations of primary factors are altered 
from one selected group to another. A simple example will serve to illus- 
trate this principle, whieh is well Keno in other contexts. "The correlations 
between height, weight, and intelligence сап be made to take widely differ- 
ent values, depending on the selection of the experimental group. If chil- 
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dren of all ages and statures are included, then the correlation between 
stature and intelligence will be found to be appreciable, This correlation is 
not spurious. It is correct to say that there actually is a high correlation 
between intelligence and stature for any experimental population of chil- 
dren with a wide range in chronological age. The taller children are gen- 
erally older, and they score higher on the tests, 
lation is limited to point age, the correlation between stature and intelli- 
gence vanishes, or nearly so. The interpretation is not to deny the meaning 
of the variables or the factors. These are the same me: 


asurements of stature 
and test score for the two groups. It is the correlations between the factors 


that are altered by the selective conditions, In the box example it will be 
seen that the correlations between height, width, and length of the boxes 
can be altered by the selection of each collection of boxes, but the physical 
interpretation of the three parameters remains precisely the same in the 
several box collections. We dwell on this point in detail because it is the 
source of misunderstanding of the factorial methods, in that some critics are 
inclined to deny the validity of the physical interpretation of factors merely 
because their correlations are altered by conditions of selection, * 

We turn now to several other examples of univariate selection with the 
same box problem, The detailed computations will not be repeated because 
the principles were illustrated with the first numerical example. The next 
few examples have been prepared to show the effects of univariate selection 
of factors, as distinguished from selection by fests, The three primary fac- 
tors for the box problem may be denoted P,, Р», and P}, These may be re- 
garded as unit vectors determined by the first three measures, 7, 2, and 8, 
in the battery, Figure 5 has been drawn to show the effects of univariate se- 
lection on the factor P. In this set of diagrams—Figures 2-10, inclusive— 
the selection variable is denoted 5, The method of extended vectors has 
been used to show the entire three-dimensional configuration in a single 
diagram in the plane of the paper. The given positions of the test vectors are 
shown by their numerical identification- 1, 2, 3, ete. The new positions 


If the experimental popu- 


* Godfrey Thomson, who has contributed fur 
variate and multivariate selec 
to whether the primary factor: 


on the problem of uni- 
alysis, has expressed doubt as 
y С and identifiable psychological 
processes, His reason is main Ween the primary factors of a 
simple structure are altered and determined in part by the conditions of selection and, 
further, that incidental factors can be ad in conditions of multivariate selec- 
tion. Our interpretation is that the primary factors represent the same basic processes in 
different, conditions of selection and that it is the correlations between these parameters 
that are altered rather than the fundamenta] meaning of the pa 


The incidental factors can be classed with the residual factors 
tions of particu 


characteristics that, have been shown for 


tion in relation to factor an 


actors. In presenting 
E o » We are giving a less pessi- 
mistic interpretation of the factorial results. 
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are denoted £z, 2r, 3r, ete. This set of six diagrams was designed by Led- 
rd Tucker to show graphically the effects of various conditions of uni- 


y: 
variate selection on the test configuration. 

Figure 5 shows the effect of partial selection on the factor P:. Note that 
the position of the selection variable S= 1 = 1: remains unaltered. The po- 
sitions of the test vectors 2, 3, 8, and 9 remain the same because they are 
uncorrelated with the factor P1. The other test vectors move radially from 
the selection variable А, as shown in the triangular configuration. The sim- 

same figure we have a diagram showing 


ple structure is unaltered. In the i 
the effect of total selection on Pi. ‘This means that all the boxes in the new 


box population have the same height. The individual box shapes will be 


determined by only 
come a constant and do 


two parameters, since one of the parameters has be- 
es not affect the individual differences among the 


Se 


ю ч 
[4 9 
o 0 2 
6x (ох $x 2x 
i 


FIGURE 


objects. Therefore, we expect the factorial result after selection to be of 
rank 2. This is verified in the diagram, which was plotted after making the 
computations that have been explained. The variable 1 disappears from 
the second analysis because it is a null vector. This is what happens осса- 
sionally when the communality of a variable turns out to be vanishingly 
small. It, usually happens when the corresponding correlations are also 
small. The new configuration lies entirely in the base line of the figure, 
Which represents only two dimensions. nu | 

If an investigator were to make a factor analysis of such a collection of 
boxes, he would find only two factors. If he ее м plot the resulting con- 
figuration, he would get à simple structure, as e nin Роша B. Here we 
see that, even though one of the three factors has been entirely eliminated 
from the individual differences and from the analysis, the same simple 
structure is identified for the remaining two factors. The interpretation of 
the two primary factors, width and length of boxes, is the same in the two 


box populations. 
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first diagram of Figure 7 it is seen that with partial selection this ambiguity 
does not arise. 

We turn next to an even more stringent selective condition, namely, the 
composite variable S = (Pit+P:+P3). This is the sum of the three box di- 
mensions. In Figure 9 the inside triangle shows the given configuration as 
before. The composite variable S is in the middle of the cor 
test 10. The new configuration, after parti 
variable S, is shown by the outside triangul 
vectors all lie in the outside tr 
the radial lines from the sele 


figuration at 
al selection on the composite 
ar configuration. The new test 
iangle, and their locations are determined by 
ction variable as before. It сап be seen that the 


o5 


o6 


E 


ласке 8 
simple structure is here 


again retained afte 
factors are the same, nı 


r selection and that the primary 
amely, those which 


are determined by tests 1, 2, and 

3. Hence we conclude that partial selection on a linear combination of even 

all the factors leaves the simple structure, as well as the physical interpreta- 
tien of the primary factors, invariant, 

However, if the boxes are üssembled 


SO as to satisfy 
selection on the composite vari 


able 5, then the 
Figure 10 we have the plot that an i 


configuration. Here it is seen th 
so that the primary factors are 
fore, that conditions of 
structure is destroyed. 


the conditions of total 
rank is reduced to 2; and in 
nvestigator would make with the new 
at the simple struct ure has been destroyed, 
no longer identifiable. We conclude, there- 
selection can be made so extreme that the simple 
In such a population the primary factors cannot be 


& Se 
Berne Selection ст P+2+8 


2 


FIGURE 9 


e o^ 
о 
о 
о 
os 
o 
o? of 


Total Selection an PeB+R 


Ficure 10 
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determined. In this case we should be dealing with a set of boxes so selected 
that the sum of the three box dimensions was the s: 
collection. For partial selection, as shown in Figure 9, the simple structure 
remains invariant. Hence we conclude that it is the total selection on this 
composite variable which destroys the simple structure. 


ime for every box in the 


Multivariate selection of subjects 


So far it has been assumed that the selection of subjects is determined by 
their homogeneity in a single criterion variable, even though this у: 
is itself a linear combination of several factors. 
made to depend on several variables 
relations of these variables must | 


"able 
Selection of subjects ean be 
, With the restriction that the intercor- 
ave certain prescribed values, In these 
cases it may not be possible to describe the correlations in the selected group 
as an alteration in the homogeneity of a single variable 
selection criterion, We then have multivariate selection of subjects. However, 
multivariate selection ean be described in terms of 

lection by the formulae already discussed, provided 
ables are properly chosen. This procedure will be 
example that has been used by ( 
variate selection, * 

In analyzing multivariate 

ables into two groups, namely. 
tion and those which are not 


,notevena composite 


successive univariate во- 
that the successive vari- 
illustrated by a numerical 
todfrey Thomson in his discussions of multi- 


selection it is convenient to divide the vari- 
, those which are directly involved in the selec- 
directly involved. The former, by which the 
selection is determined, will be denoted by subscript j and the ot hers will be 
denoted by subscript k. The given correlation matrix can then be sectioned 
in the following manner: 


The correlations of the selection tests are shown in section /? 
tion. The correlations of the same tests after selection may 
by the matrix V; which shows the new correlations that 


selection. The new correlation matrix сап then be symboli 
form as follows: 


jj before selec- 
be symbolized 
are imposed by 
zed in sectioned 


* The Factorial Analysis of Human Ability, p. 187. 
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which represents the correlations to be expected after selection. The diag 
onal elements of Vj; show the new variances of the selection tests and "al 
side entries of V;; show the covariances to be imposed by selection. 
| The problem is, then, to determine the expected correlations in the sec- 
tions 17. and Vje which is the transpose of Tj. A formal matrix solution 
to this problem has been written by A. C. Aitken and reported by Godfrey 
Thomson.* The solution is as follows: ` 


(42) Vje = Vy hi Rie 
(43) Vix = Ree = Rig Ri — КТЫ) Rx, 
(CE) Vig = Vie 


Godfrey ‘Thomson's numerical example will be used for the present dis- 
cussion to illustrate the analysis of multivariate selection in terms of sue- 


Table 11 
Table 10 Given 
5 А Factor 
Given Correlation Matrix ee 
LA M 1 124 3 4 5 6 1 
L | 1.0000 .0000 — 3741 0935 , "rn 
M .0000 1.0000 . “9274. 6416 3667 H B 
1 .374L 1. .72 „63 36 3 7 
2 |— 3n 1.00 — .56 f 132 a | Se 
3 10935 “56 10 42 35 -28 : 8 
4 "0802 “в 42 1.00 0 .24 a| 4 
5 “0668 40 35 .30 1.00 .20 : 
6 .0534 .32 ‚28 24 .90 1.00 ——— 


cessive univariate selection. His numerical example is reproduced in the 
correlations between the six numbered variables of Table 10. The columns 
Land M have been added, and they will be described presently. The given 
correlation matrix is of order 6X6, and it is of unit rank. The corresponding 
one-column factor matrix is shown in Table 11. In order to illustrate multi- 
Variate selection, Thomson imposes a change in the variances of tests 1 and 
2 and also in the covariance Ciz- The given section of the correlation matrix 


for these two variables is 


* [bid., р. 189. 
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where the unit diagonals represent the unit v 
start and where the given correlation T12=.72. 
tion matrix is to be changed by selection to 


ariances of the tests at the 
This section of the correla- 


where the two test variances are depressed to 


-36 and the covariance to 
сіг = .30. The problem іх now to de 


termine the expected covariances in the 


FiGeng 11 


rest of the 6X6 matrix and then to factor the corresponding corre| 
matrix. The direct method of Aitken can be used to solve the proble 
can also be represented as a form of successive univariate selection 

In Figure 11 the two unit vectors, 7, and Т», represent the two А Тапа 
2. They have been drawn so that the cosine of the angular separation is 
-72, which is the given correlation г. The relation between these two lonis 
which is imposed by selection is shown by the two vectors T and 7! with 
lengths of .60 and sealar product of .30 as required by the new mates V 
The two configurations have been arranged in Figure 11 to utilize еда 
metry in this case in order to simplify the transformation from one to the 


ation 
m. [t 
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other. The new orthogonal selection variables are drawn in the figure, name- 
ly, Land JM. 

The transformation from the given configuration to the new configura- 
tion can be expressed in terms of two successive univariate selections, first 
on M and then on L. The first selection on M is specified by the restriction 
that 


TAL 


p= 10974 = .6193 ; 


so that the new vectors become (7; 1) and (Ta— A). Then follows a uni- 
variate selection on L, with the value 


do 
| di = 4030, 


so that the new vectors become 

(45) Т = (Ту<=.А = В), 

(46) Ti = (Te— A+B) 5 

Which define the new configuration and the covariances. | 
The first step is to add two rows and columns to the given correlation 


matrix for L and M in Table 10. The unit vector Jf bisects the angle be- 
tween 7, and Ts, so that we ean write 


(47) M=c(T,+ Т»). 
Hence 
(48) Pim = Clr + rj), 


Where 
1 
VAL + т) 


(49) "= 


Tn the same manner the unit vector L is a linear combination of T; and Т», 


Orthogonal to M, so that 


(50) T= е(Т = Та ; 


and hence 


(51) rj = cri уа), 
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where 

1 
52 l: = -—————. 
92) V2(1 — rig) 
Having thus determined the correlations of the six tests with Land M in 
Table 11, we determine the new covariances after selection on M. These co- 
variances are shown in Table 12. The covariances after further selection on 
L are then computed in the same manner, and these are listed in Table 13. 
It is not necessary here to express Table 12 in the form of correlation co- 
efficients. 


Table 12 


Covariance Matriz after Selection on M 


L M H 2 3 4 5 6 

ee к ——— Eo m 
L 1.0000 .0000 -0668 .0534 P= goz} = 019: 
M -0000 3835 -1757 1406 
1 .3741 .3557 0.1504 
2. тт . 104 
3 .0935 . 2 ‚1350 
4 0802 2109 2 1157 
5 0668 1757 . .1687 | 0964 
6 "0584 .1406 61504 1104 11350 1157 :0962 9171 

Table 13 
Covariance Matrix after Selection on 1, 
E M 1 2 3 4 5 6 
== - E = —— A D rco M 

L .2145 .0000 .0802 —. :0200 .0172 .0143 .0115 P= gzj] =- 4030 
M -0000 .3835 .3557 ; -2109 .1757 1406 ў 
l .0802 . 2 7 .2019 1684 1347 
2 —.0802 .3557 .9 :1891 .1576 .1961 q*— 7856 
3 . 0200 .2460 2 . 1965 .1638 .1311 q = . 8863 
4 0172 .2109 “2019 -8084 1404 [1123 
5 .0143 .1757 1684 1404 .8670 0936 
6 .0115 .1406 1347 


-1123 .0936 .9149 


In Table 14 are recorded the correlation coefficients corr 
covariances of Table 13, and in Table 15 we have 
after selection, which is also represented in Figure 12. It will be seen that 
the final factor matrix is of unit rank, except for the doublet factor in the se- 
lection tests 1 and 2. The nature of this doublet factor is explained in Ta- 
ble 2 at the beginning of this chapter. The additional test is there a linear 
combination of the first two tests of the battery. Here the sume effect is 
shown because the two unique factors have been introduced into the com- 
inon factors to become an incidental common factor. The multivariate se- 
lection with the restriction that the new correlation 71? Shall have an arbi- 


esponding to the 
the final factor matrix 
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trarily chosen value of ra =.83 introduces an incidental factor, namely, the 
doublet factor II in Table 15. This introduction of an incidental factor into 
the common factors has not disturbed the single common factor, I, with 
which the problem started in Tables 10 and 11. The saturations of the fac- 
tor I have been altered, but not its identity. 


Summary 
is starts with a set of measurements for each individual 


Since factor analy 
is evidently of fundamental significance 


Member of an experimental group, it 


Table 14 Table 15 


New Correlation Matrix New Factor Matrix 


1 2 Ew ee To | pe XE 
1 87 .30 1 
2 | [83 25 2 
3 AG 21 3 
4 | -38 17 B 
t | 380) 14 2 
6 | .24 11 
| 
oo 
б 5 4 3 
مةخ ل‎ F 


Ficure 12 


to know how the factorial results are affected by the manner in which the 
iduals are selected. To the extent that a simple structure 
actors are invariant under changes in selective 
e can have confidence that the 


experimental indiv 
and its associated primary f 
conditions for the experimental group, W 
Drimary faetors represent identifiable processes whose nature transcends the 
Circumstances under which the experimental subjects happen to be selected. 
It does not follow that these identifiable and interrelated processes are 
therefore unique in the mathematical sense of being the only set of parame- 
ed for describing the dynamic system that produces the 
ance of simple structure under 


ters that can be us 
observed individual differences. The inv: 
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changes in selective conditions does imply, however, that the processes so 
identified are likely to be fruitful landmarks in a logical deseription of the 
system. А 

By univariate selection is meant the selection of a group of experimental 
subjects so that their standard deviation in a selection variable shall be dif- 
ferent from unity, which is the dispersion in a set of normalized scores for an 
initial group of subjects. The criterion or selection variable may be defined 
by one of the tests in the battery, or it may be a linear combination of sev- 
eral tests or factors of the initial analysis, If the selection criterion is corre- 
lated with any of the tests in the battery, then there will be an effect on the 
correlations between the tests in the selected group of subjects. The ex- 
pected correlations in the new group can be computed by Godfrey 
son's equation (18). The new communalities can be det 
analogous manner, 


Thom- 
ermined in an 


When the new group of subjects is as 
a selection variable L, so that its s 
equality 0245,24 1, then the new group is 
sense that varian 


sembled according to dispersion in 
indard deviation s; satisfies the in- 
ssembled by partial selection in the 
te in the selection variable is not entirely exeluded ; 
it is different from that of the initial group of subjects. When the new group 
is assembled so that its dispersion in the selection variable vanishes entire- 
ly, then its standard deviation 8:=0, and the new group is then described by 
complete or total selection on the selection variable, 

The rank of the reduced correlation matrix for the new group is the same 
as for the initial group of subjects, except when the selection 
on the selection variable, in which case the 
We then have first-order partial corre] 
variable constant, and the rank r is ther 

In partial selection when the r 
invariant, the simple structure is 
tween the prime 


but 


is complete 
dimensionality is reduced by 1. 
ation coefficients, with the selection 
1 reduced to (r— 15, 

ank of the reduced correlation matrix is 
also invariant, but the correlations be- 
wy parameters are altered. The interpretation of the pri- 
mary factors remains the same in both groups. 

In total selection, when the rank of the reduced correlation matrix is re- 
duced by 1, the simple structure may be distorted, depending on the rela- 
tion of the total selection to the primary factors. The interpretation of the 
primary factors affected by the total Selection may then be obscured. 

When the new group is selected so as to Satisfy с 
pendent selection variables, the variances of the se 
fected and also their covariances. Such д situ 
lection. Multivariate selection can be described in terr 
variate selections. The expected correlations after 
can be computed by matrix formulae derived by 
tions can also be determined by the uniy 
applied to represent successive v 


onditions on two inde- 
lection variables are af- 
ation is called multivariate se- 
ms of successive uni- 
multivariate selection 
Aitken. The new correla- 


ariate formula of Godfrey Thomson 
Mivariate selections. 
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With partial multivariate selection of a new group of subjects, the rank of 
the reduced correlation matrix is augmented by one or more incidental fac- 
tors, The number of incidental selection factors is determined by the num- 
ber of independent variables that participate in the multivariate selection. 
The simple structure remains invariant, so that the primary factors can be 
identified with the same interpretation as for the initial group of subjects. 
The additional factors may show appreciable variance, but they will not be 
invariant for differently selected groups, since they are determined by the 
conditions of selection of each group. If the attempt is made to interpret the 
incidental selection factors as basic parameters, then the interpretation will 
fail to be sustained in subsequent factorial studies of the same domain with 
differently selected subjects. The primary factors should be identified in 
the differently selected groups. 

With total multivariate selection the rank of the correlation matrix is re- 
duced by one or more factors, which depend on the number of successive 
variables on which the selection is complete; but the rank is also augmented 
by incidental selection factors, which depend on the number of variables 
ection. With total selection involving the primary 
factors, the simple structure can be so distorted that the primaries may not 
be identified. For this reason it is well to allow as much variation as possible 
domain to be investigated, thus practically elimi- 
tal selection within the parameters that are to be 
gation it is not of any consequence whether 
any of the groups of experimental subjects are representative of a general 
population. "The important consideration is that the experimental subjects 
should vary among themselves as much as possible within the domain that 


that participate in the sel 


among the subjects in the 
nating the possibility of to 
sought, In factorial investi 


is being investigated. ET "n 
these various cases of selection is very encouraging, in 


that a simple structure has been shown to be invariant under widely differ- 
ent selective conditions. Hence the scientific interpretation of the primary 
factors as meaningful parameters can be expected to transcend the widely 
different selective conditions of the objects that are measured and factorially 
analyzed. "This encouraging finding leads to a recommendation for the fac- 

When a simple structure has been found for 


torial study of any domain. 
a test battery that has been given to an experimental population and when 


а plausible interpretation of the primary factors has been found, these 
should be regarded as hypotheses to be verified by giving the same test bat- 
tery to new experimental populations that should be selected in different 
Ways. If the primary factors are in the nature of basie parameters that are 
not merely reflections of the experimental conditions or the particular se- 
lective conditions, then these factors should be invariant under widely dif- 
ferent selective conditions, and their interpretation should be the same for 
al groups. New test batteries should be constructed, 


The analysis of 


the several experiment 
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with prediction as to factorial composition of the new tests, and these should 
be tried on differently selected subjects in order to determine whether the 
interpretation of each factor as a meaningful parameter can be sustained. 

This is another situation in which the factorial methods depart from the 
conventions of statistical analysis. It is customary in statistical reasoning 
to think of a general population from which we merely draw samples, and 
the worry is then whether the sample is really representative of the uni- 
verse. In factor analysis the principal concern is to discover an underlying 
order to be described in terms of meaningful parameters which should repre- 
sent scientific concepts. The validity of a primary factor is determined by 
its fruitfulness as a scientific concept. It is inadequate if it serves merely as 
a regression coefficient. When factor analysis has completed its job of 
charting a new domain, then it may be of practical importance to establish 
norms of performance for any specified general population. Then the con- 
ventional statistical reasoning is again applicable. 


CHAPTER XX 
THE PRINCIPAL AXES 


Indirect solution 


When multiple-factor analy 


is was first developed, it consisted essen- 
tially in a shift of emphasis. ‘The dominant question in previous factorial 
studies had been as to whether the correlations could be accounted for by a 
and whether the general-factor residuals were large 
recognition of “disturbers,” which were called “group 
factors," These were sometimes frankly admitted at the start of a study, 
but always with the reserv tion that they should be regarded as secondary 
to the general factor, which was invariably postulated. With multiple- 
factor analysis the fundamental question was formulated without this re- 
striction, namely, to determine how many factors were indicated by the ex- 
‘elations. When that question was answered, the 
next question was naturally to inquire about the nature of all the common 
factors that determined the correlations. This extension of the factor prob- 
lem to any number of dimensions made it a question of fact as to whether 
of the common factors were general in the sense of participating 
in the variance of every test in the battery. 

One of the early objections, raised by Spearman, against multiple-factor 
analysis was the indeterminacy of the problem when more than one factor was 
indicated by the correlations. In the author'sinitial papers describing the mul- 
tiple-factor approach, this indeterminacy was shown to be associated with 
the question of where to locate the reference frame in the test configuration. * 
The number of dimensions of that configuration was shown to be equal to 
the rank of the table of correlation coefficients, which was defined as a square 
symmetric matrix. t 

In the author's first papers on multiple-factor analysis the indeterminacy 
of the location of the reference frame was resolved by placing the first refer- 
ence axis so as to maximize the sum of the squares of its factor loadings.} The 
next factor axis was located in the same manner by the first-factor residuals, 
and so on until the 7 principal-factor axes had been determined. Hach suc- 


single general factor 
enough to justify the 


perimentally given corr 


one or more 


* Т, L. Thurstone, «Multiple Factor Analysis,” Psychological Review, XXXVIII 


(1931), 406-27. 


+L. L. Thurstone, Theor 
1932), p. 20. 


1 Ibid., pp. 17-19, 22-27. 


y of Multiple Factors (Ann Arbor, Mich.: Edwards Bros., 
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cessive reference axis was located so as to maximize the sum of the squares of the 
factor loadings or test projections. This resolution of the indeterminacy of 
the reference frame was called the principal-azes solution.* [t has an impor- 
tant computational advantage, in that it is unique for any given correlation 
matrix. This solution was written before the simple-structure principle 
developed. The principal-axes solution has considerable theoretic 
for factor analysis, and it is of practical importance in some 
problems, 

The principal-axes solution will be described here for three 


dimensions, 
but this solution is applicable to factor problems in any number of dimen- 
sions. It will be: 


sumed, first, that the given correlation matrix /? has been 
factored into the factor matrix Fo with elements аж and that the reference 
frame of Fy is arbitrary as determined by the factoring method that hap- 
pened to be used. It may be assumed also that the diagonal elements were 
not adjusted for each factor. In this case the correlation matrix with experi- 
mentally determined coefficients represents, in general, as many factors as 
there are tests, so that the nth factor residuals are identically zero when X is 
of order nx n, The problem now is to rotate the arbitrary orthogonal refer- 
ence frame into that of the Principal axes. Later thesame solution will be dis- 
cussed as obtained directly from the correlation matrix without the arbi- 
trary intermediate factor matrix, Fo. 

Let the unit vectors in cach principal 
tion cosines of cach of these unit v 
principal axis is then identified b 
tion cosines An; and where the 
frame of Fo. The test vector j i 

The projection of the test у 
1S 


was 
al interest 
types of factor 


axis be denoted Ap and let the direc- 
ectors be denoted Amp. The first or major 
y the unit vector Aa, where p=1 with direc- 
subscript m refers to the arbitrary refere 
3 defined by the direction numbers Q jn. 

ector j on the major principal axis Ap then 


nee 


(1) лр = [PM + [V + азАзь. 


The projection of j on the unit vector А» can be denoted r 
projection is the correlation between test 
test A, of perfect reliability. 
Squaring the test projection on th 
termined, and expanding the terms 


Jap because this 
J and what may be regarded asa 


1е principal axis, Ap, which is to be de- 
for rank 3, we have 


(2) f тй» = аА, + аў, + ap + 2050 j2d1yop + 


c 
2aj53 As, 
F 2a 05s, Я 


* The solution of this problem and of the ch 
writer in 1931 by Professor Walter Bartky, 
University of Chicago. 


aracteristic equation was 


Suggested to the 
of the Department of A 


stronomy at the 


[21] 
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Summing for the n tests J, 


FA ^ ` 


x 

UE CASS - E g-X$ NA 

rip Ms “ай м, aj T № аз 
j ч 


(3) : x 
^ т o 
+ эм, V алар Mp iM алаз + Ns Nagy (jaja . 
For convenience, let 
Soe 
(4) уле e 


J 


Then the partial derivatives with respect to the direction cosine Му of 
Apis 
: duc. RS ^ AONO 
(5) дм; = Dip» а + 2) S ajaj + 2A > алајз, 
› — = — 
j j j 


and the other partials are written in the same manner. 
s of Amp are subject to the restriction that A, shall be 


The desired value 
have the conditional equation 


unit vectors. Hence we 
(6) ‚ = Mat Med MO 1 = 0. 
By Lagrange’s method of undertermined multipliers we have the following 
normal equations: 


(дш av _9 
ies zm ý 
ди Ov 
2E —=0, 
(7) \ ахз, ӘХ, 
ди дь 
EA К es) 
ТРАЕ 


ns are here the 7 values, № and 8. These are to be de- 


The (74-1) unknow " 
and the conditional equation (6). 


termined by the r equations (7) 
The partial derivatives of v are 


ov 


= 2) 
Эму Ips 
до 
(8) ie 23%, 
P ей: 


INi 
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Substituting (5) and (8) in (7), dividing each equation by 2, and rear- 
ranging terms, we obtain 


ЕЯ] Sar + В) x № > ада» F Nap Мала = 0, 
i / $ J 
(9) 4 Mp > алау + № №, ap 8) P Nap Ува =0, 
f MJ j 
| м» Малад T dep > аза al Уа + 8) = 0 
N 1 ы F; / 


This is a set of r homogeneous linear equations in the unknowns Arp, À 
Азу. The determinant of the coefficients in equations (9) must vanish, in 
order that a non-trivial solution may exist. * Setting the determinant equal 
to zero, we have the characteristic equation 


2р 


(Za?, + 8) Хада Халазз 
(10) А = | Хала; (Xa? + 8) Хаа зз =), 
aja; Xa jot ja (Sa? + В) 


It will be seen that the cocflicients аы of this determinant can be written 
in the form 


(11) [PI = |КР], 


Ж А ae 3 
where Fo is a factor matrix. The characteristic equation then becomes 


(12) [FFs + 81| = 0 

or 

(13) IP BI| =0. 

By expanding the determinant, this equation ean be written in the expanded 
form 

(14) Oil + oft hte lg i 

which is of degree r. For the present case the expansion is of third degree, 
namely, 

(15) B+ OB + 8 е = 0. 


* A trivial solution for (9) is to set all the unknowns equal to zero, 
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The cocflicients of (15) ean be written by expanding the characteristic de- 
terminant of (10), or they can be computed from the principal minors of P. 
Each coefficient ez in (14) is the sum of all the z-rowed principal minors of 
Р. All the r roots of (14) are real and negative, and they will be denoted 
B, Bo, 5 y. Be. 

Each of the roots 8, can be substituted, in turn, in (9). Each of the r 
values of 8, gives a set of direction cosines for a principal axis. When a 


p^ 


root B, is substituted in (9), the solution gives the direction cosines of Ap, 


Which are А. 

The principal axes A, are orthogonal. Their direction cosines may be ar- 
ranged to form a matrix of the transformation A from the given orthogonal 
co-ordinates of Fy to those of the principal axes. This transformation may 


be represented by the equation 


"t 


(16) Гоу = Врз 


a factor matrix for the principal axes. Each value —B, is the 
ections of the test vectors on a principal axis 
squares of the factor loadings in the col- 


where P, is 
sum of the squares of the proj 
Ay. These are also the sums of the 
umns of Fp 


Numerical example of indirect solution 

In Table 1 we have a small correlation matrix X and a factor matrix Fo 
with an arbitrary reference frame. It is desired to find the factor matrix Fp 
| "Phe first column of Fp will then be such that the sum 
lings will be a maximum. We first find the matrix 
P= ques whichis of order 2X2, since the rank of Fo is 2. The expanded form 
of the eharaeteristie equation (14) can then be written, and, sinee r=2 in 
this example, the characteristic equation is the quadratic 


for the principal axes. 
Of the squares of its loa 


(17) gd сй + е = 0. 


The coefficient e; is the sum of the one-rowed principal minors of P. These 


dios 5 е 2 g 5 
are merely the diagonal terms of P, so that 


(18) ву = L87 + 106 = 24 


The coefficient e» is the sum of the two-rowed principal minors of P. There 
is only one such principal minor in this example, namely, the determinant 


[| itself, and henee 


(19) c; = |Р] = 02133. 
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The numerical form of the characteristic equation then is 
(20) B + 2.0300 8 + 0.2133 = 0. 


The two roots of (20) are —2.855297 and — 0.074704. 


Both roots are real 
and negative. Writing equation (9) in nume 


rical form, we obtain 


=0, 


=0 


Table 1 
Factor Matriz 


Fo Correlation Matriz R Factor Matrix Fp 
| 
I II 
T a — 
2 a B 
Ж as o5 " 
4 AW iG 


t 
о 
& 
E: 
S 
i 


Matrix P = Foo 


Transformation Matrix A 


Ai Аг 
|а та I | .803525 — 595971 
2 | 1538 1:05 I | 1595271 1803525 


[P| =0.2133 [А] esci 


from which we have 


| Хи = .803525 | 


са 


(22) 


Direction cosines of major 
95271 | principal axis A; j 


where the subscript p= 1 in Anp. The major princip 
cosines Ау and №. It сап be identified in the later 
merical value of — is the 


al axis has the direction 
nt roots because the nu- 
the factor loadings of a 
az, isa maximum, is called 
axis for which the sum of the 
wan principal axis. This axis is 
actor problems. 

rincipal axis for the present numerical 


sum of the squares of 
column of Fp. The principal factor D, for which x 
the major principal axis. The principal 
squares Ya}, is a minimum is called the 5 
also of considerable interest in some f. 
The direction cosines of the mean p 
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example are computed in the same manner from equation (13), in which we 
substitute the second largest root 8». We then have 


(1.87 — 0.074704) Me + 1.33 № = 0, 


1 
с 


| 1.33 Aye + (1.06 — 0.074704) № = 0, 


N z 
from which we have 


( ке 595971 ban: Р А 
(24) je 595271 ү Direction cosines of mean 
| | principal axes Аз, 


№ = 8035 

where the subscript p = 2in Anp- 
Having found the direction cosines of the two principal 
configuration, we write the transformation matrix, А, which is 


axes of the test 


M А 
Il Жи PNE | (4-.803525) 5271) | 
(25) A= | = | 
II |] An № || | (+.595271) (4-.803525) 


est configuration ean be de- 


This is an orthogonal matrix by which the t 
1 of the arbitrary reference 


seribed in terms of the principal axes insteac 


frame I, II, in Fo. We now have 
(26) PA = Fp; 


in Table 1. It will be seen that A is an orthogonal 
The sum of the squares of the columns of A 
are normalized. The geometrical interpre ation is that the principal axes 

The cross products of the columns of A 


veetors are unit reference vectors. 
are zero, which has the geometrical interpretation that the principal axes 


are orthogonal, their scalar products being zero. 

The geometrical interpretation of the present example is shown in Figure 
7, in which the test configuration is first drawn for the arbitrary orthogonal 
axes I, II, of Fo. When the direction cosines of the major principal axis Ay 
Ao), they can be represented on the diagram, where 
Ay is shown as a unit vector. The sum of the squares of the test vector pro- 
Jections here is a maximum, as seems reasonable from the diagram, since the 
passes through the central part of the test group. On 
by measurement that the test projections on 
rical values in Fp. The second prin- 


whieh is shown numerically 
matrix, Its determinant |A| =1. 


have been found (Au, 


major principal axis 
the diagram one may verify 
the principal axes agree with the nume 
cipal axis A» is also shown in the diagram. 


The numerical values of M2 and № in (24) can be reversed in sign, since 
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the homogencous linear equation (23) defines only the ratio of Мм» to Aas. 
Their numerical values are determined by the restriction that №.+А = l. 
If the signs of Мм» and in (24) were reversed, then the second column of 
the transformation matrix A would also be reversed in sign. In Figure 1 we 
would then have the second principal axis reversed in direction. The deter- 
minant of the transformation matrix would be reversed in sign, so that 
ГА = —1. When this determinant is equal to — 1, we have an odd number 
of reversals of direction of reference axes. When | +1, there is an even 
number of reversals of axes, or no reversal, If the reference frame 7, TI is 
subjected to rigid rotation in Figure 1 so that I goes to Ay when II goes to 
As, then the determinant of the orthogonal transformation is +1. If there is 
an odd number of rev 


s, this determinant is equal to = 1, Such is the 


case if we reverse the second column in A so that the reference vectors be- 


z 
1 


FIGURE 1 


come A; and —Л». It can be seen in the figure that the frame I, II cannot 


move to A; and — As by a rigid rotation within the dimensionality of r=2, 
For most factor problems the differentiation bet ween the cases in which 
|A| = +1 and |A| = —1 is of no practical consequence. It is of some in- 
terest to know the relation between the ambiguity of signs in equations (23) 
and the geometrical interpretation of the problem. 


A direct solution for the principal axes 

In previous sections of this chapter we h 
mination of the principal axes, in that the correlation matrix was first, fac- 
tored into a factor matrix Fo with an arbitrary reference frame which de- 
pended on the particular method of factoring that was used on the correla- 
tion matrix. That was the author's solution when the principal axes were 
first described. The principal-axes solution can be written more directly in 
terms of the correlation matrix and without an intermediate factor matrix 
Fo. 


Consider the correlation matrix 7? 


ave considered an indirect deter- 


jx and the desired factor matrix P, of 
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Table 2. In this table the correlation matrix has only four rows and four 
columns, and the factor matrix has two factor columns. It is desired to lo- 
cate the first principal axis so as to maximize the sum of the squares of the 
test vector projections on that axis. This means that the sum of the 
of the factor loadings ад in the first column of F must be 
that the first-factor residuals must be minimized. The fir: 
can be written 


squares 
maximized or 
st-factor residuals 


(27) Tik = 


адак = py, 
where pj; is the residual. Squaring the residuals, we have 


(28) Tjk — 2ajy + айай, = рўк { 


and, summing for all correlation coefficients, we have 
ND S. а NS 
(29) > S Tjk = 25 N E y Yaa. = WO NO pj. 
== = پک‎ а Ч — — l": 
J d j k i k k 


For convenience, let 


(30) b =e. 
j k 
Then 
дг C E 
(31) düs = 4È rnan + daa > аһ : 


k 


* 2 y artis /ativ (S (з n 7 i 
Setting the partial derivatives equal to Zero, we obtain 


(32) Erga, = алаў. 
In the rest of this chapter the notation 8 will refer to the latent roots with 


positive signs. Let the sum Уай = B, which is the largest latent root. Writ- 
ing (32) in matrix form, we obtain 


(33) Ra = aß, 


where /? is the correlation matrix, а is 
namely, the first latent root. 

Equation (33) is not directly a computing formula, but jt ied. 
on which Hotelling's iterative factoring method is based, * 


a column vector, and B is a scalar 
элу 


equation 


* Harold Hotelling, “Analysis of a Complex of Statistical V, 


۴ ariables i "incip: 
Components," Journal of Educational Psychology, XXIV (Septem! vito Principal 


er and October, 1933). 
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Hotelling's iterative method of factoring 
‘This method of factoring enables us to obtain the desired factor matrix Fp 
s directly from the correlation matrix without the inter- 


for the principal axc 
Hotelling's iterative method of factor- 


mediate arbitrary factor matrix Fo. 
ing starts with a trial column vector из, 50 that (33) becomes 


GM) Ruy = ti, 
where the product vy is also a column vector. The next trial vector, us, is 
taken proportional to ti, SO that us kvi. Then 


(35) Rus = vs, 
Which gives the new trial vector, Us »», for the next trial. The stretching 
factor [is arbitrary for each trial. This process continues until the product 
numbers v, are proportional to the next. preceding trial numbers u, to any 
The column vectors wu; and v, are then both 


required degree of accuracy, 
Let the proportionality be 


proportional to the desired column vector а, 
represented by 


(36) и = pa, 


so that Ru=v becomes 


(37) R(pa) = (pag) . 


The scalar product (w) then is 


= (pa)(paB) = 1780 = pp. 


(38) (uv) = 
Hence 
, зав 
(39) ses шыла, 


vom V pig 


which reduces v to the desired column vector ау. 
determined at the solution by the simple rela- 


The latent root 8 ean be 


tion 
Xv 
(40) B= Su 


have been determined by equation (27), 


When the first-factor residuals 
the residual correlation matrix ољ can be factored by the same process to 
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give the second column of F» whose elements are аў. The sum of their 
squares is 


(41) 


which is the second latent root. This is Hotelling's iterative method of fac- 
toring the correlation matrix. It is an improvement on the writer's original 
method of finding the principal solution by the indirect method of first 
finding an arbitrary factor matrix Fy and then rotating the reference frame 
to the principal axes. The principal-axes solution can be written to repre- 
sent the whole factor matrix F, instead of one column ata time, as was done 
here. Then the interpretation of the characteristic equation and the latent 
roots are the same as in the writer's origin 


axe: 


al paper on the principal axes, 
Hotelling has renamed the principal-axes solution the principal components. 
Sometimes this term refers to the factor loadings 0,5 in Fp and sometimes 
to Hotelling’s iterative method of factoring. This method and the writer's 
original indirect method give numerically identical results, as will be seen 
in the accompanying example. In computational work the indirect solution 
requires less labor with the computing methods now available, but the di- 
rect solution is theoretically the more inte 

Hotelling restricts himself to the case of unit diagonals in the correlation 
matrix, but his method is applicable when othe 
orcommunalities, are written into the diagonals. 
that the rank of the correlation mat rix, the 
the latent roots are markedly 


resting. 


r values, such as reliabilities 

It should be carefully noted 
number of common factors, and 
affected by the diagonal values. The rank of X 
is lowest, when communalities are used in the diagonal cells; When unit 
diagonals are written, the rank generally becomes equal to the order of the 
correlation matrix, 


An important characteristic of Hotelling's iterative method of factoring 
is that the convergence is slow when the larger latent roots are nearly the 
same. The convergence is rapid when the latent roots differ markedly in 
numerical values. In the numerical example of this chapter we have chosen 
a case in which the latent roots are numerically quite different so that the 
convergence is rapid and the solution is reached in a few bs The reader 
should not expect such rapid converge 


nee m many practical problems in 
which the roots may have nearly equal numerical values, 


Numerical example of Hotelling’s iterative method 

In Table 2 we have a small correlation matrix of order 44, whose latent 
roots are widely separated so as to give a rapidly convergent example of 
the iterative process. The first trial vector, "a, is taken Proportional to 
the column sums of 72, so that the largest numerical v. 


1 OR; alue of u, is unity. 
The matrix multiplication, Ru,—w, is shown in 


the next column. The 
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reciprocal of the largest entry in column ri is recorded as the arbitrary 
stretching factor Ку and we then have i2 = kr, in the next column. The ma- 
trix multiplication, Aus rs is recorded in the next column. The proe- 
ess terminates when the two successive values из and us are found to be iden- 
teal Within the required degree of accuracy, 

Pho reduction of te lo the desired column vector ау is shown under the 
table. Tire scalar product (wets) is computed and recorded as shown. The 
square root of this number is p8, as shown. Finally, the reciprocal 1 p8 is 
computed and recorded. ‘The final column ауд is computed from re by the 
relat lon ay, =v, pp 

Phe first-factor residuals are 
Section of Table 2. The same proc 


then computed and are shown in the lower 
ess of factoring is applied to this table, 

ya as shown. Finally, the two column 
Vectors ayy are recorded in the desired factor matrix Fp. The first trial veetor 
fora residual correlation matrix ean be taken proportional to the absolute 
Sum of the residuals in each column, including or excluding the diagonals. 


One A : б 
ne can also justify taking as trial values the square roots of the diagonal 
residuals, 


m he factor matrix for the principal 
1¢ sum of the squares of each columr 


roo o 
t. Further, the eross product of eac 
|i column 


wi i 
ith the resulting column vector d; 


axes has two properties of interest. 
vof the factor matrix £j is a latent 
h pair of columns is zero. We then 


laye А 
We for the sum of the squares of ea 


(12 R 
) N ap = Bos 
J 


t ; A. 
"i for the cités produet of any pair of columns, p and q, 


(43 ^ 
) N'ajaj, = 0. 
d 

1 * T. З 2 

oth these properties can be easily verified numerically in the factor ma- 
trix p 

р. 

l'he present example does not show the problem that arises when the con- 

gence is slow. The wide separation of the latent roots in this example 

nee shown here. When the latent roots are 


Produce : 
luces the rapid converge 
to introduee other de- 


lH. ais : A 
a arly alike, the slow convergence makes it desirable 
HEGE 5 : : 
Ces to increase the convergence to the desired solution. 
Th 
e H 
s Squaring process 
tie ae the iterative me 
SCC 2 a 
th оп a correlation matrix mM W 
» Same numerical values, the process converges slowly. The labor required 
reach the solution is then so considerable as to be discouraging. The con- 


< solution is 


thod of factoring for the principal-ax 
‘hich the largest latent roots are of nearly 
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vergence can be increased by the squaring process described by Hotelling as 

an improvement on the first form of his iterative factoring method. * 
Consider the first iteration Fu; —v (equation [3 H), with a column vector 

ну Which is either arbitrarily chosen as a start or chosen by inspection of the 

correlation matrix X. If, now, this product v; is used as the next trial vector, 

we have 

(44) Rv, = о. 


Substituting (34) in (44), we get 


(45) (Кир) = vy 
or 
(46) Ruy = ve, 


Using р» as the next trial vector, we get 


(47) Гоз = v, 
and then 

(48) Rv; = ow. 
By (47) this becomes 

(49) (оз) = v, , 
and, by (46), 

(50) Itu, = ру, 


This process can be continued, so that 
(51) Ru =v, 


where x denotes any power of Је, If R is raised to a sufficiently high power, 
the product vector v, is proportional to the desired solution ал, even if the 
arbitrary trial vector is badly chosen, A reservation may be made about the 
choice of the trial vector u. If this trial vector is exactly Proportional to any 
column of Fp, including the secondary roots, the equation (51) is satisfied, 
This situation is not likely to happen in practice, so that the iterations will 
practically always lead to the major principal axis, A simple example of this 

* Harold Hotelling, “Simplified Calculation of Principal Components,’ 


’ Psychometrika, 
1 (1936), 27-35. 
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effect in which iteration does not lead to the major principal axis is the fol- 


lowing 2X2 correlation matrix: 


1.00 — 80 +1 
— .SO 1.00 +1 
R и 


Iteration with the trial vector (+1, +1), as shown, leads to the mean prin- 
cipal axis, whereas iteration with the trial veetor (+1, —1) leads to the 
major principal axis. Examples of this kind arise very rarely with experi- 
mental data, because it is very unlikely that a trial vector will be propor- 
tional to one of the secondary columns in F,. 

From (46) it is evident that one iteration on R? gives the same result as 
From (50) we see that one iteration on P24 gives the 
ison R, and so on for any power of At. The more 
set to some extent by the labor of 


two iterations on K. 
same result as four iteration 
rapid convergence of this process is off: 
lation matrix by itself several times, and it must also 
act that a large number of decimals are car- 
its full effectiveness. 


multiplying the corre 
be considered in relation to the f 
ried in this process in order to retain 
The relation of this process to the characteristic root 8 can be seen if we 
Write Ra — ag (equation (33), where a is the desired column vector and 8 is 
the largest latent root of R or of a residual matrix. This can be written 


(52) 


from which we have 


(53) 1 (5 ка) EN 
80 that 
(54) Ез Ra = а. 


Repeating this process gives 
(55) Ka = Bra, 
from which it is seen that if X is raised to the power 2, the iteration with a 
gives the product numbers B*a. 
An interesting property of the 
to a sufficiently high power it appro: 
of approximation.* The factor matrix corr 
rarchy and Its Properties,” Psychometrika, 111 (1938), 


correlation matrix is that when it is raised 
aches unit rank to any required degree 
esponding to such an augmented 


* Cyril Burt, “Phe Unit Hier 
151-08, 
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correlation matrix has only one column. This symmetric matrix Л, which 
is of unit rank, can be factored by a single-column factor matrix, which may 
be represented by the column vector Ё„= А, so that 


(56) Rt = AA’; 


and it is of some interest to see the relation between the single-column factor 
matrix A of ? and the first-column a of the factor matrix Fs: Let 


(57) а= рд, 

and substitute in (55), which becomes 

(58) R*(pA) = B*(pA) ; 
and this can be written 

(59) PRA = pgzA ; 
but R* can be factored by analogy with (55), 
(60) RA = КА , 
where k is the largest characteristic root of R, namely, 
(61) Е= Л'А. 
From (59) and (60) we get 

(62) РКА = pg*A , 
and hence 

(63) Е =p, 

We can then re-write (60) in the form 

(64) REA = grad. 
From (56) we find 

(65) AAA = BA, 
and by (57), 


a p 
(66) p? Uds c p Ba; 
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and, since B=a’a, 


(67) Ва 
p p 

so that 
(68) - 1 

р 
Hence by (57), 

1-2 

(69) а = | = КА, 


so that the desired column vector ад of R can be obtained from the single- 


This is not the usual procedure in the squaring proc- 
edure is to continue the squaring unt il 
227, are proportional to 


factor loadings of R7. 
ess. The generally recommended proc 
the column sums of two successive squarings, / to 1 
any required degree of approximation. The column sums of А, or numbers 
proportional to them, are then used as a trial vector u on the original corre- 
lation matrix X, so that Ru=v. The column vectors wand v may then be 
expected to be proportional, so that u or v is proportional to the desired 
column vector ад. If proportionality is not attained, the iteration may be 
continued on R. The actual reduction from v to a is done by noting that the 


ratio 


(70) 


and if one of the trial numbers t4 is unity, then 8 can be read directly from 
the corresponding value of vj. If one of the trial numbers 1; is not unity, the 
root B can be determined from the ratio 8— X|v| /X| «| for the last iteration. 
A useful device, especially when it is desired to reduce v to a unit vector in 
the configuration before reaching the final principal-axes solution, is to com- 


pute the desired loadings from the relation 


s 3 
(71) aa = tj = س‎ 
У Хит 
Where 
a 1 
(72) (m o 
У Хир 


When the correlation matrix has been raised to some power т, SO that the 
column sums of A are proportional to the column sums of the next previous 
matrix 722/? to the required degree of approximation, then Z2? isof unit rank 
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to the same approximation. The single-factor loadings a; of R= can be found 
by the centroid method, which gives the unique values 


کڪ چ )73( 


7 ^st; 1 


in which 


tj; = sum of absolute values of cell entries in row j of RF y 


m;= +1, according to the sign of б}, 


sj = algebraic column sums of Rë. 


The desired factor loadings ayı can be 
of R* without an additional iteration on R. If the column sums S, of /2?, or 
the single-factor loadings a; of R7, are used in the iteration Ru=v, then 


it is expected that u and v will be proportional and that these will also be 
proportional to алд. Therefore, we write 


obtained from the column sums s; 


(74) 


а = pu. 
Let s; denote column sums of R. Then 


(75) “su = Т, 
where T' is the column sum of the 
this column sum T to the column su 


Hence by equation (70), 


product numbers Ru = V. The ratio of 
m Zu is the largest characteristic root 8. 


= y 

(76) E 
From (75) we have 

— Уз _ 
(77) = 


From (74) we have 
(78) Sa = f = pixa 


so that 


(79) 2 


and hence 


(80) p= VE 
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By (74) we then have 


(S1) a= 


where wis proportional to the column sums of R= and sı represents the corre- 
sponding column sums of X. It is good computational practice to verify the 
determination of aj, by the iteration Ra = Ва for any method of obtaining the 
values of a. An alternative procedure is to carry out the iteration Ru=v 
and then reduce v to aj, by equation (7D). Several relations have been de- 
seribed here, but it is not necessary in practice to verify all of them on each 
numerical problem in finding the solution ау. The same procedures are used 
on the correlation matrix residuals for determining each column of the factor 


matrix Fp. 


Numerical example of the squaring process 
In order to illustrate the squaring process, we shall use a small example 


which does not converge so rapidly as the previous example with Hotelling’s 
original iterative method of factoring. This example is shown in the factor 
matrix JF, of Table 3. The two characteristic roots are 1.26 and .33 as shown, 
esent principal axes, as can be seen from the fact that 


The two columns repr 1 | 
Жадау» =0. The corresponding correlation matrix X is shown in the same 


table, | 

The expected column sums 8; of R? are computed from X, and, since these 
are not proportional to the column sums of Ze, the complete matrix X? is 
computed, Under each matrix we have reduced the column sums s; to the 
proportional values Кау, in which the largest value is unity. This facilitates 
comparison of successive sets of column sums. 

The expected column sums of R are computed from Л, and these are 
again not proportional to those of 2°, so that R4is computed in full as shown. 
Next we compute column sums of RS and compare with column sums of +, 
These two sets approach proportionality, but they still differ in the second 
decimal, Hence J is computed as shown. The column sums of 0° agree 
closely with those of /?%, and hence it is not necessary to compute R" in 
full. The discrepancies here do not exceed .0002. 

In Table 4 we have a summary of numerical examples. The trial values 
u= ks; from R“ are used in the iteration Ru=v, as shown. The computation 
Yn =3.2400000 leads to the values of ал by equation (71). 

In the same table we have the computation of aj; from column sums of 
R In the first column of this table we list u= ks; of R". The values of aj; 
are then obtained from equation (81). For check purposes these values 
Should be verified by the iteration Ita = Ва for one or more rows of R. 


08 


Table 3 
Factor Matrix Fp 


Correlation Matriz R 


| 1 ul 1 2 3 4 
1 Т .0 1 -35 —.28 
2 p 1.6 = 4 2 .14 .20 
3 E —.4 3 41 —.24 
"UU т 4 21 17 
>а ‚2 —,7 5j 14 16 .38 m 
Za? | 1.26 .33 ks; .37 .42 1.00 —.39 
Matriz R? 
1 2 3 4 
-2184  —.1239 
1.0000 —.5673 
Matriz Rs 
1 2 3 4 
1 -98018424 — . 84015792 . 70013160 E .56010528 
2 — .84015792 -72588528 — . 59436288 . 47865276 
3 . 70013160 — .59436288 . 50584392 — .40151268 
4 — .56010528 -47865276 — .40151268 .32041953 
8; -28005264 — - 22998276 .21009996 — 16254567 
ks; 1.00000000 — .82121261 -75021596 — .58041113 
Matriz RS 
1 2 3 4 
1 —2.11759599 1.76466333 —1.41173066 
2 * ) 1.81515047 —1 .51250038 1.21003780 
3 1 . 7646 333 —1.51250038 1.26054199  —1.00839609 
4 —1.41173066 1.21003780 —1 .00839609 .80670750 
зу . 70586534  — .60490810 . 50430885  — .40338145 
ks; 1.00000000 26977 -71445476 — 157147083 
1; 7.76451864 5.54610179 4.13687205 
Matriz Rs 
8j 4.48421281 —3.84361097 3.20300917  —2.56210732 
ks; 1.00000000 — .85714285 -71428572  — .57149857 
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The principal-axes residuals 

When one factor column of Fp has been determined, the extraction of the 
next. faetor from the correlation matrix is usually done by computing the 
residual correlation matrix, 
(82) R, = В – аа, 
where X, denotes here the first-factor residual matrix from X. These resid- 
uals ean be computed in the same manner as for any other factoring method. 
yy . " . . М Y а 6 d 
Che residual correlation matrix is then used for determining the next factor 


Table 4 


Computation of аз by Iteration Ru = V 


1 | 1.00000000 1.26000000  .тооооооо Suv = 3. 24000000 
2 | — -85714285  —1.08000000 — < 60000000 1 w= .30864198 
Sr s “90000000 -50000000 1 VSw= 355556 
4 | — 157142857  — .72000000  — . 40000000 


Computation of а from Column 
Sums u = kss of R5 


u ай 
1 1.00000000 . 70000000 Xu = .28571430 
2 — .85714285 — . 60000000 Zu 57142857 
3 50000000 Es .36000000 
4 — — .40000000 p ‚48999998 


‚70000000 


column ау of Fp. The procedures on R, are the same as those which have 
been described for the first factor. 

In addition to the customary method of computing residuals, we shall de- 
Scribe two methods, namely, (1) Hotelling's adaptation of the squaring 
Process to the computation of residuals and (2) Tucker’s method of comput- 
Ing the subsequent principal-factor loadings without computing residuals. 

If the squaring process has been used in the determination of the first- 
factor loadings aj: of Fp, then the augmented correlation matrix № can be 
Used also for writing the residual correlat ion matrix raised to the same power 
With considerable reduction in computational labor. For this purpose we 
Shall first prove several useful relations. 
Let 


(83) 
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where Q is a square symmetric matrix of unit rank produced by the column 
vector a, which has been determined. The residual matrix Ry is then 


(84) = Р – 0) = В – аа. 


We also have the two relations previously established, namely, 


(33) Ra = Ва, 
and 
(85) B= daa. 


From (33) and (83) we have 
(86) RQ = Raa’ = Ваа’, 
(87) RO = BQ.. 
From (83) and (85) we can write 
Q* = (aa')(aa') 
= ala'aja’ 
= ава’ 
= Baa’ 


(88) Q = BQ. 


Applying these relations to the augmented correlation matrix Rê, 
have, starting with R? and using (87), 


we 
RQ = RRQ) = [80 = BQ 

= BR(RQ) = BR(BQ) = BRQ 

= B(BQ) 


(89) R3Q = BQ, 


and, in general, 


(90) RQ = BQ. 
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Another relation involving X7 can be shown from (87) and (88) by writ- 


ing 
RSQ = RRQ) = IPQ" 
= R(RQ)Q = REQ 
(91) RQ =RQ, 
or, in general, 
(92) 10) = RQF. 


Powers of Q can be reduced from (88) by writing 
Qt = QQ? = QBA) = 80°, 
(93) Q = 80800) = BQ, 
Ог, in general, 
(94) Q: = BQ. 
The residuals ean be expressed by (84), (87), and (88) from the squaring 
Process by writing 
jp = (R — Q)? = Л? — 2R + Q* 
= qp — 28Q + BQ, 


(95) pp = 0 — BQ, 


9r, in general, 
(96) Re = RF —В*10, 


by which the residuals FR; raised to the power x сап be obtained from the al- 


ready computed matrix R7. In using (96) the procedure is as follows: 


1) The column sums s of IF are used as trial numbers in the iteration 
Rs=p. Several iterations are taken, if necessary, until aj; are de- 
termined. 

2) Determine characteristic root 8. 

3) Compute the product numbers №ға = 8а, in which 


Bra _ pe, 


a 


Table 5 


Correlation Matriz Ra 


Factor Matriz Fp 


Г 


1 4 1 I H Hr 
1 Tu тр Tha тц au V/A i 1 ап аг 05 
2 Ta Tee To Ta anV d 3 d a e 
3 ти тз T3 Ta aa V dii i| аа ae аз 
4 та To та та ауді І Увїо 0 
І au Vai an VBı ã алув. і aa VÊı i =8 = 
Correlation Matriz Ry 
ад 
1 2 3 4 == 
1 .04 .04 .08 —.10 1 .0 Bio .81 
2 .04 .29 18 08 2 3 Vg-.90 
3 .08 .18 .56 16 3 .6 
4 | —.10 .08 .16 161 4 6 
Correlation Matriz Ry 
а; 
1 2 3 4 F سسحت‎ 
1 | .04 .04 .08 —.10  .00i 1 E 8= .49 
2 04 .290 18 0з әт; 2 2 Vg-.70 
3 | .08 .18 56  .16 154 3 4 
4 | —10 (08 .16 ‘61 ‘54 4 | —.5 
I | .00i 2: 5и 5и —81 I `0 
Correlation Matriz Ra Factor Matriz Fp 
js a: ze 
1 2 3 4 I II ——— гп III 
1| оз .04 .08 —.10  .00i 14 1| .0 ТЛ am 0 
2| 04 29 .18  .08  .27i 147 2| 4 2| oh A 
3] 08 .18 .56  .16  .54i  .28i з|—.2 3|.6 .4 =2 
4 |—.10 .08 .16  .601  .54 —.35i 4| .0 41.6 —5 0 
I| 00; .27i .54i  .54i —.81i .00 I| .0 
II | di dài .28; —.35i .00 —.49 I| .0 
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4) Determine 877! by the division 

: 8* 
= P 
din. 


5) Compute / by equation (96). 


The squaring process has advantages in rapid convergence, but it is also 
quite laborious in practice, especially when large correlation matrices are 
to be raised to a high power. 

Another solution for the principal-axes problem without residuals has re- 
cently been proposed by Ledyard Tucker. * It is ingenious and looks prom- 
ising, but it has not yet been tried extensively. In Table 5 we have a 4X4 
correlation matrix 2, which gives the first principal-factor loadings in the 
adjacent column ал. The sum of the squares of this column is 6:=.81. 
When this first factor has been determined, the usual procedure is to write 
another matrix of correlation residuals. The present method avoids the 
Necessity of computing residuals. Instead, the original correlation matrix is 
merely bordered, as shown in literal notation at the top of Table 5. The in- 
dicated elements of the bordering column and row have the effect of reduc- 
ing the largest characteristic root to zero. Then the major principal factor 
of the bordered matrix 5 is the second principal factor of the original ma- 
trix 2). The corresponding factor matrix Fp is also shown in literal form at 
the top of Table 5 with an added row. The sum of the squares of the first 
column of this factor matrix is identically zero, so that the major principal 
factor now becomes the second column of Fy. 

The bordered matrix Rs is shown in numerical form in the same table. 
Hotelling's iterative process on this correlation matrix gives the second- 
factor column ау, аз shown, with the second characteristic root 8= 49. 
The matrix X; is constructed by bordering 72, with two additional rows and 
columns. Hotelling's process on this mat rix Ёз gives the third-factor column 
а. The three factor columns are assembled in the numerical factor matrix 


Fat the bottom of the table. 


Graphical aids in factoring the correlation matrix 

In the iterative process of determining the principal axes the computa- 
tions ean be reduced by graphical methods, at least in the first few trials.t 
The iterative process depends on the relation 


(97) Ru = 051°, 


* The Determination of Successive Principal Components without Computation of 
Tables of Residual Correlation Coefficients,” Psychometrika, IX (September, 1944), 
49-53. 

+L. L. "Thurstone, “Graphical Method of Factoring the Correlation Matrix," Pro- 
ceedings of the National Academy of Sciences, XXX, No. 6 (June, 1944), 129-34. 
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where 2 is the given correlation matrix, н is the trial column vector, and v 
is here a column vector proportional to the product Ru. Consider in particu- 
lar the scalar product of row j of Rj and the trial column vector u. This 
scalar product is 


(98) R pyu = 


from whieh we have 


> 

Qt _ Ryu 

(90) 0; = 525; 
zw 


but this is also the well-known equation for the slope of the regression line 
ron u through the origin for a plot of r against u. If we plot columns of у 
against the trial values tg, we get а plot with л points. The best-fitting 
straight line through the origin (regression r on u) can be drawn by inspec- 
tion, and the slope of the line is easily read graphically with sufficient ac- 
curacy for the first few iterations. That slope is the value rj. To plot, say, 
thirty points and to draw by inspection the best-fitting straight line through 
the origin takes less time than to obtain the cumulative sum of thirty eross 
products on a calculating machine. 

By equation (97) it is evident that at the solution the values v= Uj = ау. 
When a set of values of v has been determined graphically, we plot v against 
u and find the slope m. In the first few graphical iterations, this slope will 
not be unity beeause the trial values ware probably too large or too small. 
If the true values are better represented by ku, in whieh K is a stretching 
inst ku will give a slope of e k instead of v. When 


factor, then a plot of r ag | 
the slope of v against Ки should be unity. 


these graphs have been drawn, 
But the obtained slope of v against u is m for the values actually used. 
Hence the observed slope” 18, Having found the slope m of the plot of 


v against u, we find k= У m. Then if we should take a new set of trial num- 
bers r; = uy, the iteration would give, instead of equation (99), 


(100) 


Where у=. The slope of y against г should now be unity, and the values 
of y should be used as the trial veetor for the next graphical iteration. The 
determination of m and & can be done with a slide rule, and the new trial 
Values y=v/k can be similarly determined. The slope m ean be found cither 


by inspection or by simple summing with the method of averages, in which 


mM=v/Su for like signed pairs of v and u. 
A numerieal example is shown in Table 6, which starts with the correla- 
tion matrix 2 of order 10X10 and rank 3. The corresponding factor matrix 


1 for the principal axes is shown in the same table. It is the solution to be 
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found from IR. The latent roots of R are 2.85, 1.47, and 0.63. These are the 
sums of the squares of the columns of Fp. In the same table the initial trial 
vector u for the major principal factor is taken roughly proportional to the 
sums of the columns of /t. Column 1 of R is plotted against column и, and 
the slope is estimated to be approximately +.05 and is recorded in column 
vı. Ten such plots give the values in column vi. The slope is read directly 
from each graph by noting the ordinate of a straight-line fit at w;=1. Next 
obtain the two column sums, Xu; and Xe, The ratio Sv /Su= m, and 
k= Vm. This summation can be absolute sums for like signed pairs of u 
and v. Then compute yi=v,/k with a slide rule. These are also the trial 
values ue for the next trial. Proceed likewise for three trials, which give the 
desired values of ад to two decimals. 

The first-factor residuals are then computed. These are Pik = т алак. 
Choose as a starting vector that column of the first-factor residuals which 
has the largest absolute sum, ignoring diagonals. This is column 1. Its val- 
ues are recorded in column u; for factor II. The procedure is now the same 
as before. The third iteration was taken on a calculating machine, and it 
gave the second-factor loadings ау», the second column of Fp to two deci- 
mals. The last iterations for each factor can be done on a calculating ma- 
chine to obtain greater accuracy, while the first few iterations can be done 
graphically to save time and labor. The computation for the third column 
of Fp was done in the same manner. 

Occasionally, when two or three latent roots are nearly the same, the iter- 
ative process will be found to oscillate, or the convergence will be slow. "This 
is an indication that the test configuration has nearly equal thicknesses in 
these dimensions, so that the configuration is nearly circular or spherical in 
these dimensions. If the purpose is to extract the maximum variance from 
the correlation matrix, it does not matter where we place a set of orthogonal 
axes in those dimensions which are represented by nearly equal latent roots. 
In such a situation it is desirable to reduce the obtained vector v to a unit 
vector in the system without waiting for complete convergence. That can 
be done with a stretching factor c on v so that cv=a, where a are the desired 
factor loadings. The value of ¢ can be found from equation (72). By this 
device the computer need not be unduly delayed by slow convergence in 
obtaining a factorially useful solution. The graphical method here described 
can be applied to a variety of computational problems. 


Additional theorems 

We shall consider here some additional properties of the principal-axes 
solution which are of theoretical and computational interes 
with the relation of equation (32), we write 


Starting 


(101) Hag = айв, 


* The theorems in this section were assembled by Ledyard ‘Tucker, 
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for the first column ад of the factor matrix F, for the principal axes. For 
the second column we have, similarly, 

(102) лар = ajb, 

and, in general, 

(103) Ёла} = AjpBp - 

The elements of Fg are а and the characteristic roots 8, may be taken as 
diagonal elements in the diagonal matrix D. Then, in matrix form, 


(104) RF, = FD. 


The matrix /’, is orthogonal by columns, so that > аз = 0, where р and q 
J 

represent any pair of columns. Since У а=8 the columns of Fp can be 

1 

normalized by the postmultiplication 1,073. The matrix F D= is then an 

orthogonal matrix, whieh may be denoted A, 

(105) poss oœ Р = АЮ, 

by which we sce that the columns of A are proportional to the columns of the 

factor matrix Fp From (104) and (105) we have 

(106) КЕ,0 = F,DD^, 


and hence 
(107) RA = AD. 


Premultiplying (107) by A’, we have 
(108) ARA = ААР ; 


but, since A is an orthogonal matrix, we get 


(109) NRA = р 
ог 
(110) В = ADA’. 


Since AA'— Г, we get from (105) 
(111) Fr = р. 


aracteristie roots 8, are as follows: The 


Several useful relations of the ch 
factor matrix F is the communality A7. 


Sum of the squares of each row j in any 
Hence the sum of the squares in the whole factor matrix F is equal to the 
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sum of the diagonal elements in e. The sum of the squares in each column p 
of Fis the root 8,. Hence the sum of the squares of all the elements in F, 
is equal to the sum of the roots 8,, and this is also the sum of the diagonal 
elements of fe. 

The sum of the squares of the correlations in R is equal to the sum of the 
squares of the entries in the matrix P= FF, where F is any factor matrix 
for R. We write 


Р М? 
(112) Tik > Э nml; 


т 


and then 


cA RES 

(113) ъ= € NC X auus. 
Ра ak L JmU km ; 
j k j k m / 


which becomes 


—\ ее ky “ ^ ^ 
(114) > Mn = > 2. ( Daman) ( 2 һам | 
man m M j n 
or 
(115) S NW SN SS us NB 
بک‎ a T — —Ó ہے‎ E 
1 k m AN j 
but 
(116) P=F'F, 
in which the elements are 
ү - чар 
(117) Рам = > js ум у 
J 
and hence 
ds 5 EQ ái 
(118) > Dhs > Ў. 
Jj k m M 
For the principal axes we have, by (105), 
(119) ЕЁ» = (AD) (AD!) 


= DNAD. 
But A is an orthogonal transformation, and hence 


(120) Р = р =P 


ps 
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Hence the diagonal elements of P are the characteristic roots of X. It can 
be shown that the characteristic roots of 7? and P are identical. 

If, in applying Hotelling’s iterative method of factoring, the first trial 
Vector is orthogonal to the first s principal axes, it can be shown that all suc- 
ceeding trial vectors will also remain orthogonal to the first s principal axes, 
and they will approach the (s+1)th principal axis. 

It ean also be shown that if we subtract a number I: from each diagonal 
clement of 2, then the characteristic roots are reduced by the same scalar А, 
but the solution A of equation (110) remains unaltered. The introduction 
of the decrement k on the diagonal elements of X can be used for accelerat- 
ing the iterative process. The iterations can be made to approach the mean 
Principal axis instead of the major principal axis by this device. The mean 
Principal axis can also be determined by iteration on 777, which has the 
characteristic roots 1/85, and these are the diagonal elements of 07". 


Advantages and limitations of the principal-axes solution 

The principal-axes solution for the factor problem has an intrinsic inter- 
est because of its many mathematical properties, including the fact that it 
isa unique solution for any given correlation matrix. That was the writer's 
motivation in the original formulation of the principal-axes solution in 1932. 
Boon after publication of the solution, it was realized that the solution did 
Dot satisfy the psychological requirements for which the simple-structure 
Solution was devised. Because of the rather general confusion about the con- 
troversial issues in this problem, we shall summarize briefly the advantages 
And the limitations of the principal-axes solution, including a description of 
the types of problem for which the principal-axes solution is applicable and 
Of those to which the principal-axes solution is not applicable. 

A factorial analysis ean be made for one of two purposes, namely, (1) to 
Condense the test Scores by expressing them in terms of a relatively small 
number of linearly independent factors or (2) to discover the underlying 
ünetiona] unities which operate to produce the test performances and to 
describe the individual differences eventually in terms of these distinguish- 
able functions. Many arguments about factorial methods could be resolved 
if the participants would indicate which purpose they have in mind, because, 
Strictly speaking, there is no intrinsically correct factorial met hod except in 
terms of what the investigator is trying to do. According to the first purpose, 
actor analysis becomes essentially a statistical method of condensing the 
A to the second purpose, factor analysis becomes essen- 
aining or rejecting hypotheses about the 
at produce the test scores. These two 


test Scores, According 
tially a scientific method for st 
nature of the underlying processes th 
Points of view are both legitimate in their respective places. There is trouble, 
however, when two investigators have these different points of view and 
Proceed to argue about factorial method without recognizing that they im- 


504 MULTIPLE-FACTOR ANALYSIS 


ply entirely different objectives with their analyses. It is the second objec- 
tive that has guided the writer, since 1931, in developing multiple-factor 
methods. It is the first objective that is usually implied by mathematical 
statisticians who write on factorial problems. This difference in objectives 
is largely responsible for controversies about factorial method. Many mis- 
understandings in this field would probably be resolved if we all recognized 
that a factorial analysis should be judged in terms of the purpose for which 
it is made and that multiple-factor analysis is a flexible and powerful tool 
that can serve as a guide in a variety of scientifie problems. 

Let us consider, first, a table of scores s;; for individuals i in tests j. This 
is the score matrix of order n ХУ. For convenience let the scores be recorded 
in standard form, with unit standard deviation and with origin at the mean. 
This table shows n scores for each of the N individuals. If the tests are cor- 
related, we may undertake a factor analysis in order to condense а large 
score matrix so that each individual may be described as to the same domain 
in terms of a relatively small number (r) of linearly independent factor 
scores instead of the large number (n) of test scores with which we start in 
the score matrix. This is cs 


ntially a statistical problem, and, as such, there 
is no demand that the factor scores shall have any particular interpretation 
in terms of the processes that produce the scores. This situation is very 
similar to that of writing a regression equation in which we want a set of co- 
efficients for the independent variables which will give that linear combina- 
tion of them which minimizes the residuals in estimating the dependent 
variable. When we solve the least-squares 


problem and write a regression 
equation, we demand only that the regression coefficients shall be numbers 
which minimize the residuals. We usually make no demand that the regres- 
sion coefficients shall be interpreted in terms of the processes which pro- 
duced the dependent variable scores. If that is ever done, we can expect 
each of the regression coefficients to be complex in nature, in that it is af- 
fected by all or by many of the processes which affect the problem. How- 
ever, it is just as legitimate to ask fora condensation of the test scores into a 
relatively small number of indices to replace the original test scores as it is 
to ask for a linear combination of the independent variables for the purpose 
of making the best-possible prediction of the dependent variable, without 
any reference to the question of what the regression coefficients might mean 
in terms of the processes that fundamentally determine them. 

Starting with the score matrix of order nN, it is our object, according to 
the first purpose of factor analysis, to write another seore matrix of order 
r XV in which r <n, and we want to deseribe each individual in terms of this 
smaller number (r) of indices so as to cover essentially the same domain 
which is covered by the larger number (л) of tests in the whole test battery. 
The fundamental factor theorem which was first formulated in 1931 applies 
here, namely, that the number of linearly independent factors in a correla- 
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¢.* The correlation matrix 


tion matrix is the rank of the correlation matri: 
for this statistical problem has unit diagonals because the correlation ma- 
trix R= (1/N)SS', where the elements of S are standard scores. 

The score elements in S are fallible, in that they are experimentally de- 
termined, and hence there is at least some part of the variance of each test 
which is fortuitous and attributable to variable errors. The rank of the cor- 
relation matrix with experimentally determined correlation coefficients is 
sarily of rank equal to the order of the correlation matrix. Hence, 
strictly speaking, п =r with experimentally determined correlations. If we 
carry out the factor analysis with this literal interpretation, then we shall 
have as many faetors as there are tests. We might as well use the tests as 
they stand for the description of each individual, since we have gained noth- 
ing in the economy of deseription. However, there is a gain in the statistical 
sense in that the new faetors would be uncorrelated, whereas the tests are 


correlated, The investigator may consider it worth while to use a set of n 
uncorrelated factor scores for each individual instead of a set of n correlated 
test scores. "hat may be his choice. 

Кеопошу of description of each individual can be obtained in most prac- 
tical problems, so that r is less than л, even with the present formulation, 
because it may be possible to select a relatively small number of factor 
Scores or indiees which cover most of the variances of the tests in the bat- 
tery, This is, in fact, the objective when the principal-axes solution is used 
Without rotation of axes. The investigator then makes a factor analysis in 
terms of a relatively small number of factors which account for the test 
Scores with residuals that are so small that he is willing to ignore them. The 
number of factors to be used in such a description depends on the size of the 
residuals that are judged to be negligible. 

In the writer's first attempt to solve this problem, the first column of the 
factor matrix was so determined that it accounted for the maximum possible 
amount of the variance in one factor. Geometrically, this solution is found 
by placing the first axis in the test configuration so as to maximize the sums 
ОЁ squares of the test vector projections on the first axis. This is, in fact, the 
Major principal axis. The next column was so determined that it accounted 
for the maximum possible amount of the variance from the first-factor resid- 
uals. The process can be continued until the residuals are negligible. It is 
Evident that the principal-axes solution is exactly the desired solution for 
this problem. At first, it was necessary to factor the correlation matrix with 
An arbitrary orthogonal reference frame, which was rotated to the principal 
axes. Then, only the first few principal axes were used to describe the test 
battery, leaving the remaining variance unaccounted for in the form of resid- 
uals. With the publi sation of Hotelling’s iterative method of factoring, it 

* L. L, Thurstone, “Multiple Factor Analysis,” Psychological Review, Vol. XXXVIII, 
No. 5 (September, 1931). 
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became possible to determine the principal axes directly from the correla- 
tion matrix, one at a time and starting with the major principal axis. The 
factoring need continue only until the residuals are judged to be negligible. 
The solution is in such a form that we can say with confidence that no other 
set of r linearly independent factors can be determined for the given set of 
test scores which accounts for more of the variance. In this statistical sense 
the first r principal factors constitute the best description that ean be found 
for the test scores with this number of factors. In practice, there is a limita- 
tion of excessive labor with a large correlation matrix. А practical compro- 
mise is to determine a factor matrix with one of the centroid methods and to 
determine one or two more factors than would be required by the principal 
axes in order to cover the same total variance. This centroid frame can then 
be rotated to the principal axes when that is the desired solution, 

We consider, next, the second purpose for which a factorial analysis сап 
be undertaken, namely, to discover the underlying functional unities which 
operate to produce the test performances, A verbalism is frequently given 
as an objection to the mere statement of this purpose, namely, that no meth- 
od of analyzing data can discover factors Let us meet such a triviality by 


acknowledging that no method of analyzing data can ever discover апу- 
thing in science or give any interpretation of the data. Discovering any- 
thing or interpreting experimental findings of any kind always implies the 
subjective functions of a human being, the experimenter. Analytical meth- 
ods can aid his thinking, but they never write the interpretations for him. 
The analytical methods that he selects depend on his purposes 

an example of the second purpose, namely, to discover functional 
unities and their nature, let us consider a set of tests of visual perception. 
The correlations between the tests might indicate that more than one factor 
is involved and hence that visual perceptual efficiency cannot be adequately 
described as a single function but rather as a complex of several or of many 
functions. On the basis of previous factorial experiments, two visual func- 
tions, such as visual acuity and visual perceptual closure, might be postu- 
lated among many other factors. Every visual test will involve visual acuity 
to some extent in its variance; but, if the figures to be seen are reasonably 
large and distinct and sufficiently illuminated, the factor of visual acuity 
will be represented in only a small and negligible part of the total variance 
in studying individual differences in such tasks. Similarly, the factor of 
visual perceptual closure will be represented in most visual tasks, but most 
of the tests may have little or almost none of their variances accounted for 
by individual differences in visual closure. If such factors operate to produce 
individual differences in performance on visual tasks of considerable variety, 
it should be possible to predict with some confidence that the relative vari- 
ances will be large or small in these two factors by merely examining the 
psychological nature of each test. If the battery of visual perceptual tests be 


n 
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considered with special reference to these two postulated factors, apart from 
the other factors that are undoubtedly present, then we shall have a fan- 
shaped configuration of test vectors in the plane of the two postulated 
factors. ‘Toward one edge of the fan we should find those tests which have a 
large part of their variance attributable to one of the postulated factors, 
and toward the other edge of the fan we should find those tests which are rel- 
atively more saturated with the other postulated factor. This is, of course, 
the simple-strueture principle in two dimensions. If it is found that the test 
; really do lie in such an arrangement in the test configuration and if 
their relative positions agree with the psychological interpretation of the na- 
ture of the tests, then we shall have experimental evidence that sustains the 
factorial interpretation; but here, аз elsewhere in science, no hypothesis can 
ever be proved to be correst by any experiment. It can only be shown to be 
plausible. If, on the other hand, the test vectors do not oblige us by falling 
into the predicted fanlike structure but fall, instead, into some other entirely 
different arrangement, then we have evidence that our postulated factors do 
hot agree with the functional unities that operate to produce the test per- 
formances. We must then guess again as to the possible nature of the under- 
lying order, ‘This is, in fact, the rule in factorial experiments, They usually 
tell us, in effect, that we must guess again. When a factori: 
sustained, it is a challenge to inv stigate further with the leads whose plausi- 
bility has been indicated factorially. An example of such a failure by the 
Writer was a hypothesis that the general intellective factor was a matura- 
tion factor, Faetorial results at different age levels have recently shown 


vector: 


hypothesis is 


is probably wrong. 
ents it is well to design the tests so that the 


that such a hypothesi 

In setting up factorial experim 
factorial results may have an opportunity to give us crucial tests of a hy- 
Pothesis or in the choice of rival hypotheses. One such pattern of great use- 
fulness is to design tests which clearly include the postulated factor and 
Other tests in the same domain, as similar as possible, which do not involve 
Jance. Presence or absence can then be 


the postulated factor in the test у: 
Predicted for the postulated factor in cach test. Those tests which do not 
involve the factor should lie at one edge of the fan in two dimensions, in one 
Plane in three dimensions, or in a hyperplane in n dimensions. Furthermore, 
All the tests which do involve the postulated factor in their variances should 
All lie on one side of the hyperplane and not on both sides of it. This is 
again the simple-structure principle in analyzing the test configuration. 
Scientific interest in a simple structure, when it is found, is relative to the 
State of knowledge at the time of the experiment. For example, if we should 
Set up some tests of visual perception and some tests of auditory perception, 
We should not be surprised to find that the visual tests had something in 
common which was entirely absent in the auditory tests. We might then 
call the factors merely visual and auditory, and that would be a first rough 
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guess, which would, no doubt, be refined with further experiments. That 
there should be distinguishable functional unities for these two modalities 
is probably taken for granted, so that the rough factorial experiment would 
be of little interest. We should know such a result before we started. On 
the other hand, the clean separation between the verbal factor V and the 
word-fluency factor W was not at all evident when several crucial tests were 
devised to test the hypothesis that at least two such factors were responsible 
for some of the individual differences in verbal tasks. Two synonyms tests 
were devised, with the prediction that one would be high in V and zero in W 
and that the other synonyms test would behave in the opposite manner 
factorially. The prediction was sustained, and this constituted a lead in the 
further investigation of verbal functions in human intelligence. 

Now suppose that the factorial analysis was made by determining the 
principal axes and that the solution was left in that form without rotation 
of axes to simple structure. The test scores would, indeed, be accounted for 
by the smallest possible number of factors, and the residuals would then be 
minimized by the well-known method of least squares. In the two-dimen- 
sional example the major principal axis would be in the middle of the fan of 
test vectors, and the second principal axis would be arbitrarily at right an- 
gles to the first axis. Such a resolution does not give us the information we 
are looking for if our purpose is to investigate hypotheses concerning the 
underlying processes that determine the individual differences in perform- 
ance. It should be evident that the principal-axes solution is the ideal solu- 
tion for the problem of statistical condensation of the test scores into the 
smallest possible number of factors but that this solution i inadequate for 
the analysis of the underlying processes and their identification. 

We have considered the principal-axes solution with reference to two dif- 
ferent purposes for which a factor analysis may be undertaken. There are 
several situations in factor analysis in which the principal axes are of inter- 
est, apart from the major purposes for which the analysis is made. Even 
though we discard the principal axes as the final factorial resolution to be 
studied for scientific interpretation, the principal axes are, nevertheless, of 
fundamental interest. Consider the problem of analyzing a correlation ma- 
trix with experimentally determined coefficients. Because of the fact that 
every coefficient in the table is determined in part by fortuitous variable 
errors, we must recognize that the total variance cannot be accounted for in 
terms of a limited number of factors whose interpretation may constitute 
the major purpose of the analysis. A part of the variance must necessarily 
be assigned to the variable errors. The first task is to factor the correlation 
matrix in terms of a limited number of factors, and the second task is to ro- 
tate the arbitrary reference frame to that of a simple structure, if it ean be 
found for some of these factors. In determining the first-factor matrix be- 
fore starting the rotation of axes, it is desirable that the given number of 
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factors shall account for as much as possible of the variance of the correla- 
tion matrix. If we could disregard the practical problems of labor, time, and 
cost, we should analyze the correlation matrix by determining the successive 
principal axes with communalities in the diagonals. These would be repre- 
sented by the successive columns of the factor matrix. When the residuals 
became small enough to ignore, the factoring would be discontinued, and the 
factor matrix would then be ready for the rotational problem. We should 
sonable amount of 


like to have the assurance, if we could get it with a r 
labor, that the r factors that have been determined account for as much of 
surance would be rep- 


the total variance as is possible with r factors. ‘That as 
resented by the principal-axes solution for the first r principal factors. But 
it should be noted that such a principal-factor matrix is not the final solu- 
tion if the purpose of the analysis is to discover the nature of the underlying 
Order, Such a principal-factor mat rix is only the starting-point for the rota- 
tional problem. All students of factor analysis are therefore interested in 
the principal-factor solution, no matter what their point of view may be 
as regards the purposes of factor analysis. Fortunately, this computational 
problem can be dodged by using one of the less laborious factoring methods 
to obtain a factor matrix with arbitrary orthogonal axes, continuing the 
factoring one or two factors further than would be required by the principal 
axes. The rotational problem would then give essentially the same solution 
as would be obtained if the principal axes had been known. No matter what 
ed, it is a safe rule to continue the factoring until 


Method of factoring is us 
one is sure that the factoring has gone far enough. Too many factors can do 
no harm, but too few factors are sure to cause trouble in identifying the 
Structure, If too many factors are determined in the factor matrix before 
rotation of axes, then the residual factors appear in the rotation of axes, and 
they are left without interpretation. When the principal-axes solution be- 
comes available with less computational labor, it will, no doubt, be preferred 
by all students of this subject, and they will start the rotational problem 
With the principal-factor matrix. 

„ Another situation in which the principal factors are of interest is found 
11 the rotational problem, When the rotational problem begins, one can be 
reasonably sure that the test configuration is thin in one or more directions 
that represent residual factors which are to be ignored in the interpretation. 
Tt would be useful to be able to determine the mean principal axis of the 
test configuration at the start of the rotational problem. One of the residual 
actors would then be indicated. The next principal axis might also be negli- 
Bible as to variance. In this way several residual factors might be deter- 
Mined at the start of the rotational problem if the mean principal axes could 
idual planes can be identified in the rotational 


2€ readily computed. The res 
abor might be avoided if the mean principal 


Procedure, but some of this 1 
axes could be readily determined. 
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For the first purpose in factorial analysis the principal-axis solution is 
indicated, and for the second purpose the simple-structure solution is indi- 
cated. Other solutions may be added eventually for both of these problems, 
especially if non-linear factorial methods are developed; but the present 
discussion is limited to the linear factorial methods that are now known. 
The writer developed both of these solutions, first, the principal-axes solu- 
tion, which was discarded for the simple-structure solution because of psy- 
chological considerations. Most mathematical statisticians who have con- 
tributed to factorial theory have limited themselves so far to the principal- 
axes solution and its properties. This is natural because the principal-axes 
solution lends itself well to mathematical treatment. It is to be hoped that 
mathematicians will turn their attention also to the challenging problems of 
developing the simple-structure concept and its implications. It has chal- 
lenging implications for the development of scientific methods that are by 
no means limited to psychology. 


CHAPTER XXI 
THE APPRAISAL OF INDIVIDUAL ABILITIES 


In previous chapters the main problem has been to devise factorial meth- 
ods which would be sufficiently powerful to deal with the complexities of 
multiple factors so as to reveal functional unities, with a minimum of restric- 
tion as to their nature and the interrelations of the factors. To discover the 
factors and to sustain or refute postulated factors have been the principal 
purposes. The development of the factorial methods has been a continuous 
Process of compromising between the theoretically complete and ideal solu- 
tions and those solutions that can be made available with a reasonable 
amount of labor, time, and cost. We turn, now, to the problem of apprais- 
ing individuals as to the factors that may have been identified, either as 
reasonably pure factors or at least as functional unities that are practically 
useful in dealing with individual differences. 

In dealing with this problem of appraising individuals as to their profiles 
Of abilities, we shall develop, first, what seems now to be a rather complete 
factors that have been identified and which 
1 consider the compromises that will be 
allow a complete solution. 


statement of the solution for the 
can be tested for; and then we shal 
Necessary in practical situations which will rarely 


The regression x on 5 

Let us assume that a set of л tests is available for which the faetorial eom- 
positions are known and that the whole battery of tests is given to each sub- 
lect for the purpose of writing his profile of r factorial abilities. Here, as be- 
fore, let the standard score of individual in the primary ability p be de- 
Noted vp; Similarly, let the standard score of individual i on test j be de- 
Noted в. The problem then is to estimate the standard factorial scores Tp: 
In terms of the given and known test scores sj;. This is a conventional form 
Of statistical problem which calls for the regression ть; on sji The given 
test scores are the independent variables in terms of which the individual 
factorial scores are to be estimated. We then have the equation 


(1) ты = Dd wisi + épis 


J 


Where the subseript p refers to primary factors, j refers to tests, i refers to 
Individuals, and w is the regression coefficient or weight of the score sj; in 
test j in the estimation of the primary factor p. The residual or discrepancy 

etween the true value of xp: and the best value that can be obtained as a 
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linear combination of test scores is denoted ej," It is assumed here that the 
true individual factor scores are the values гь. 

Equation (1) is a regression equation, in which the test scores sy, are the 
n independent variables and rp: is the dependent variable to be estimated by 
the best linear combination of the test scores. The problem of finding the 
best linear combination of the test scores is that of finding the regression 
weights i5; for tests j in estimating the dependent variable which is the 
score rp, in the primary factor р. The weights w are to be so determined as 
to minimize the discrepancies. The discrepancies can be written by (1) in 
the matrix form 


(2) Nowa Bs 


where X of order иж № is the matrix of factor scores with elements ты, W of 
order r Xn is the matrix of regression weights Wpj, S of order n XN is the 
standard test score matrix with elements sj; and £ of order r XN is the ma- 
trix of residuals in which the sum of the squares of each row is minimized. 
The problem can be solved separately for each primary factor p. Then in 
equation (2) the matrices X, WS, and £ are reduced to row vectors of order 
LXN. Then it is the п values of wp; in the row vector W of order 15x n which 
are to be determined so as to minimize the residuals. * 

For each primary factor p that is to be estimated in the subjects, we have 
a residual ej; for each individual. These residuals are the № elements in the 
row vector J. The sum of the squares of these residuals is given by the ma- 
trix product £E’, which is a scalar. From (2) we have 


(3) Nh’ = (X = WS)(X = WS)’, 

which becomes 

(4) ER! = XX = XSW — WSX' + WSS'W", 
Since these terms represent scalars, we сап write 

(5) EE' = XX’ — WSN’ + WSS'W'. 
l'rom this we obtain the normal equation for W, 

(6) NS’ = WSS’, 


Multiplying both sides by L/.V, we have 


rere ЯУ ае 
T x Хе = "(x sw). 


* H. W. Turnbull and A. C. Aitken, Canonical Matrices (London: Blackie & Sons, 
1932), p. 173. 


= ————— 


TIE APPRAISAL OF INDIVIDUAL ABILITIES 513 
But 
(8) = is = Йу» 


where „у is a row vector whose elements are the correlations between the 
test scores sj, and the primary-factor scores vpi These are the correlations 
between the tests and the primary factor p, and hence they are, in a sense, 
the validity coefficients of the tests for the primary factor p. Similarly, 


(9) = SS!’ = в; 


where 2, is the correlation matrix with unit diagonals.* From (7), (Sand 
(9) we get 
(10) Ro = И; 


and from this we have the desired equation, 


(11) № = Wor, 


where Wpr (Cor wy) аге the desired regression coefficients in equation (1). 
Equation (10) has interesting interpretations for statistical and psycho- 
logical problems. In order to determine the regression weights w, we need 
the inverse, Ry. This inverse does not exist if Ry, is singular. Hence the 
Correlation matrix Rj with unit diagonals for the independent variables 
must be of rank equal to its order. The number of terms in JS of the re- 
gression equation (2) is equal to the number of independent. variables. 

If the situation should arise in which a set of supposedly independent 
Variables is not linearly independent, then the inverse, Rj, could not be de- 
termined, In dealing with experimental data it may happen that the inde- 
Pendent, variables owe their linear independence to fortuitous variable 
errors, If the variable errors were then eliminated, the set of supposedly in- 
dependent variables would no longer be linearly independent. If, in spite of 
such a situation, the regr ssion equation were computed with as many re- 
Sression weights as there are supposedly independent variables, we should 
expect to see absurdities in the regression weights. They would express only 
the fortuitous variable errors. This situation is not unlikely when a regres- 
Sion equation is written in terms of a large number of independent variables. 

The difficulty with correlated independent variables whose true rank may 
be less than the order can be avoided by an application of multiple-factor 
analysis. The correlation matrix for the independent variables should be 


* The unit diagonals are called for here because we are now dealing with a convention- 


al regression equation. 
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factored with reliability coefficients in the diagonals. These diagonals in- 
clude not only the common-factor variances of the independent variables 
but also those parts of the unique variances which the independent variables 
may share with the dependent variable. These diagonals exclude, however, 
the unreliabilities of the independent variables. When the residuals are 
judged to be vanishingly small, the number of factors is also the number of 
terms which may be written legitimately in the regression equation. 

Having determined the number of terms which can be written in the re- 
gression equation, one proceeds next to arrange the independent variables 
in the same number of groups as there are number of terms in WS, which is 
also the rank of the independent variables. The grouping should be done so 
that closely correlated variables are in the same group as far as possible. A 
new set of independent variables can then be written as centroids of the 
groups. These, in turn, constitute the independent variables Rj, in equation 
(11), with unit diagonals for determining the r weights Wp. * 

In application to a psychological problem, equation (11) gives results 
which are of theoretical, as well as of practical, interest. Suppose that the 
number factor N is to be appraised by a large battery of tests which repre- 
sent several factors. If the instructions for the number tests include some 
variance on the verbal comprehension factor V, then this factor will have 
small saturation on the number tests. In writing the regression equation for 
the appraisal of the number factor, one might find a small negative regres- 
sion weight on the verbal factor V. The interpretation then would be that 
the number factor N is a linear combination of the number test composite 
AN, and the verbal test composite V, in which the number test composite 
Nais coplanar with N and V, and between them. It is as if the number 
factor N would be determined from N and V by subtracting a correction for 
the verbal component in the number tests. This is illustrative of the system 
of weights that may arise in a complete solution for primary-factor scores 
as linear combinations of the tests in a diversified battery. 

Much thought has been given by some students of factorial theory to this 
problem of appraising the primary-factor scores of individuals on the basis of 
their scores in a diversified test battery. The solution to this problem is giv- 
en by the regression equation (1) and equation (11) for the regression coeffi- 
cients, but in practical application it is rarely feasible or even desirable to 
use the theoretically complete solution. 

The regression equation (1), or its variants, presupposes that a complete 
test battery has been given to each subject, and it implies that the battery 
represents all the relevant primary factors. In practice it is desirable that 
the tests for each factor be as far as possible self-contained, so that a subject 

* This is the problem which was discussed in a different setting by Ragnar Frisch in 


his monograph, Statistical Confluence Analysis by Means of Complete Regression Systems 
(Pub. No. 5 [University of Oslo Economic Institute, 1934]). 
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can be examined for one factor without corrections for the other factors 
which may be irrelevant for the occasion. In order to prepare tests to satisfy 
this requirement we merely select several tests which have high saturation 
on the desired factor, and the score in such a composite may then be used in 
practical situations as the subject's factor score. The additional refinement 
that could be brought into the appraisal by giving the tests for all other 
known factors in order that a regression equation might be solved for each 
individual for each factor is rarely, if ever, worth the extra trouble. In most 
situations this complete procedure would even be absurd. 

In selecting the subtests which are to constitute a composite test for any 
primary factor, it is useful to distribute as far as possible the variances on 
the factors which are irrelevant for the composite. Thus in subtests for a 
composite to appraise the space factor S, the several subtests would prob- 
ably be found to be impure, in that each of them would have some variance 
attributable to verbal comprehension V in the test instruction, to the num- 
ber factor N if a part of the task induces numerical thinking, and so on. 
Now, if all the space tests had an appreciable secondary saturation on the 
Verbal faetor V, then evidently the composite would represent a linear com- 
bination of S and V with major variance in S. This effect is reduced by se- 
lecting the subtests so that the irrelevant variance is scattered among differ- 
ent factors. 

When a test composite for a primary factor has been selected, it can be 
used as a test for the factor in question quite independently of test com- 
posites for other primary factors that may or may not be given to the same 
Subjects. Tt should be specially noted that the correlations bet ween the test 
Composites for the several primary factors will generally have higher inter- 
correlations than the primary factors themselves. Further, if the primary- 
factor scores are estimat 
diversified test battery, the 
expected to be lower than the correlations between the corresponding test 
composites. Nevertheless, the problem of appraising an individual as to 
each of several factors is best solved in the practical situation by using a test 
In theoretical studies of the intrinsic correlations 
able to use the more complete factorial meth- 


ed by the regression equation (1) for a factorially 


correlations between such factor scores may be 


Composite for each factor. 
between the factors it is prefer 
ods previously described. 

We turn next to the other regression, namely, to predict the test scores 


8j; when the individual factor scores 75; are known. 


А 
The regression s on x 

This regression implies that the primary abilities of an individual are 
known and that it is desired to estimate what his performance will be on a 


test with known factorial weightings. This is the reverse of the previous re- 
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gression z on s, in which it was assumed that an individual's scores are 
known and that his primary abilities are to be appraised. 
The case in which sj; is to be estimated by т can be written in the form 


(12) Sji == E WipTpi + 6j, 


p 


in which the notation is the same as in the previous case. By similar reason- 
ing we get, for the regression coefficients, 
(13) їй» = lt. 
The matrix Ij, is of order nXr, and it shows the correlation between each 
test j and each primary factor р. These correlations are, in a sense, validity 
coefficients, because these are the best available correlations between the 
tests and the factors that the tests are intended to appraise. The matrix 
It] is of order 7 Xr, and it is determined from the intercorrelations Ry. The 
matrix Rp, has unit diagonals because it shows correlations between factors 
of unit variance in the common-factor space. Since the primary abilities are 
linearly independent, it follows that the rank of /2 is r and hence that Tis 
is non-singular. 

If the primary abilities are uncorrelated, the matrix It, becomes an 
identity matrix and its inverse is then also the identity matrix. The regres- 
sion weights then become 


(14) Ws = Ry = F (orthogonal ease) ; 
and in this case the regression weights are, in fact, the elements of the factor 
matrix F, whose elements show the correlations between the tests and the 


orthogonal unit reference factors. The corresponding regression equation for 
the simplest case in which all the contributing factors are common then is 


(15) Sji = Гут (orthogonal case) , 


which is also the fundamental observation equation with which we started 
the development of multiple-factor theory. 
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configuration. on 


r selection, 447, 453, - 
ince matrix, 4 


acceptability of 
factorial method, 361, ¢ 
factorial solution, 2 
hypotheses, 187-88, 2 , 201, 
determining reference vectors, 334 
diagonal entries, 284 
factorial invariane 
#rouping of tests 
, method, 171 
Intereolumnar correlation, 260 -62 
intereolumnar proportionality, 260-62 
maximum variance, 177 
reference frame 
analytical, 377-78 
configurational, 178 bil 
graphical, 108, 176-79, 195, 334-36 
statistical, 178-81 
reflection of test vectors, 1 
Simple structure, 334-40, 35: 
single factor by Spearman, 
tetrad, 271 
, Unique configuration, 178-81, 293 
Titerion 
in зш of minimizing weighted sums, 
399 
prediction of, by regression equation, 68 
of science, 52 
variable, in selection, 443-53, 458, 464, 
_ 470 
Crossroads in the Mind of Man, by Tru- 
man L. Kelley, ix, 203, 309 


360-63 
in multiple group 


4, 165-66 
56, 377-78 
260-62 


би 
Iv 
= 


Curve-fitting, 368 
Cylinder problem, 117-124 


Data 
characteristics of, 375-7 
of facto 57, 63, 101 
factorial interpretation of, 378 
fallible, 101, 260-61, 300—301, 505, 508 
treatment of, 177, 375-76 
Degrees of freedom, 52 


Dependence (see also Independence) 
experimental, 441-42 
lin » Linear dependence 
Derivatives, partial, ¢ A75 


ıs test, 366 


Determi 


с 
of composite matrix, 302-4 
defined, 5 

diagonals of, 3 

evaluation o 
of hierareh 
of a matri: 
minors of, 
order of, 3 
of product of two matrices, 20 
properties of, 3-13 

term of, 4-5 

of transformation matrix, 40, 479-80 


Diagonal 


Diagonal ent 
communalities as; see Communality 
in correlation matrix of unit rank, 259, 

262, 274, 277-81, 284 

criterion for determination of, 284, 287 
geometrical interprets tion of, 137-88 
in Heywood case, -90 

i ive method of fac- 


ve, 290 


posi 
-axes solution, 177-78, 484, 


in principa 
509 
and rank, 82, 282-87 
reliability coeflicients as, 83 ot 
in Spearman's single-factor method, 274, 
284 
of 
greater than communalities, 286 
greater than unity, 262, 289-90 
high and low, 284-87 
less than communalities, 286 
less than unity, 284, 286 
unity, 98, 176-77, 284-87, 201, 484 
as stretching factors, 132 
in tetrad difference equation, 284 
unique factor contribution to, 79, 320 
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Diagonal entries—continued 
unknown, 110-16, 149-67, 283 
variable, 287 


Diagonal matrix 
defined, 21 
types of, 21-23 
use of, 132, 137-38, 351-53 
Diagonal method of factoring, 101 
Dickson, L. E., 1, 44 
Dimensionality (see also Rank) 
augmented, 421-22 
of configuration, 91-92, 194 
of first-order domain, 411—114 
loss of, 204, 239-40 
of second-or domain, 411 21 
of total test space, 176, 448 
Direction cosines 
defined, 34 
of normal 
to hyperplane, 3- 
349-51 
to score surface, 323-24 
positive, 216, 342 
in positive region, 342 
of primary vectors, 136-38, 228, 351 
of principal axes, 477-79 
property Of, 
of reference vectors, 88-43, 132. 38, 195 
205, 227, 347, 384, 474-77 
cale in plotting configuration on 
sphere, 132-37 
in transformation matrix, 3843 
of unit test vectors, 137, 349 
Direction numbers, 50, 227, 383-85, 308, 
474 
Dispersion, measures of, 368 
Distribution of ability, 62-66, 325, 307 60 
Disturbers of tetrad equation, 273, 473 


Domain 
alternative simple structures in sume, 
320-31 
defined, 55, 320 
of factor analysis, 55-57, 324-25, 340, 


3, 175, 


35, 226-27, 238-39, 


as 


375-76 

parameters of, 71-72, 177-81, 313-14, 
320, 413 

relation between first- and second-order, 
411-34 


of statistical method, 291 
underlying order in, 55-56, 258, 320-25, 
360, 472, 506-7 
Double-group method of rotation, 408-10 


Doublet 
defined, 182 
in estimation of communalities, 299, 
306-7, 315 
occurrence of, 363, 441-42, 468 
second-order, 417-18 


Dresden Arnold, 1 
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Elements 
of correlation matrix, 71-73; sec 
Correlation coetticients 
of diagonal matrix, 
of factor matrix, 71 
loadings 
of matrix, 2 
notation for, 1: 
of population matrix, 68, 71, 356 
of score matrix, 68 
unitary, 343-44 
Ellipsoid, projection of test. vectors into, 
4441-45, 448 
tion 
ucteristie; ser 


also 


see also Factor 


Characteristic equa- 


tion 

fundamental factor theorem, vi, 77-81, 
149 

fundamental observation, 68, 71, 325, 


7—90) 


normal form of, 34 35 
notation for 

expanded, 13, 16 

matrix, 19 

rectang » 14-15, 17-18 

summational, 15-16, 28-31 
of oblique simple structure 
Otis-Kelley formula, 449-5 
partial correlation formula, 448-50 
of plane, 34-35 

il, 52 
regression 

s on x, 515-16 
s, 511-17 

Tor score surfaces, 322 

simultaneous, 13-18, 201 

of slope of line, 33, 499-500 

of test vector, 148 

tetrad-difference, 265, 273, 293 

of trace, 226-27, 241 

veetor, 234, 230 
Error 

experimental, 146 

random, 73, 83, 261 

sampling, x, 260-61, 264, 283 

standard, 261, 267 

uncorrelated, 73, 83 

variable, 73, 83, 286, 306-7, 505, 508, 

513 

Error factor, 73 -75, 83-86, 146 
Error-factor variance 

and correction for attenuation, 146 

notation for, 85-86 

жена of, 73-75, 83—85, 286, 505, 

508 


226, 4175 


INDEX 


and reliability, 81 
separation of specifie factors from, 73, 
S4, 286 
Essential Traits of Mental Life, by Tru- 
man L, Kelley, viii, 287 
Experimental dependence, 441—12 


Factor analysi 
ussumptions in; sce Assumptions 
British school of, 273 
data of, 57, 63, 375 
istribution of ability, 62-67, 325, 
» 
criteria of acceptability in 
domain of, 55-57, 314-14, 
76 
and empirical prediction, 59-62, 287, 


e Criteria 


exploratory nature of, 56-5 
287, 320-27, 332-34, 340, 
and faculty psychology, x, 56, 70, 145- 
46, 177 
first approximation in, 54-5 
fundamental concepts in, 328-29 
fundamental equations in, 68-86, 149 
fundamental postulates of, 62-63 
ideal constructs in, 70 
ideal solution in, 179 
methods of; sec Factoring methods; Ro- 
tation of axes _ 
und multiple correlation, 59-02, 68, 179, 
327 
parameters of; see Paramet 
planning of, 176, 313-14, : 
purpose of, 55, 62, 70, 92, 
325, 340, 348, 360, 375, 471-72, 
as n scientific method, 319, 338 
Steps in, 101, 283 
texts in, v, viii-ix | 
Uniqueness of solution in, 
180, 329-34 
Factor A nalysis, by Karl J. Holzinger and 
Harry H. Harman, viii 
Factor A nalysis to 1940, by Dael L. Wolfle, 
ix 
Factor axis, 120-21, 473; see also Reference 
frame 


92-03, 178, 


Factor loadings 
augmented, 131 
centroid, 152-54, 157 

170, 279 
common-, 73-86, 146 
contribution to variance of, 68-86 
defined, 71, 73 
effeet of ы; 
adjustment of planes on, 249-51 
selection on, 360, 369 
RUE 146 
extended, 225 
geometrical interpretation of, 89, 99 
in Heywood case, 289-90 
invariance of; see Invariance 
maximum variation in, 36 


‚ 161-65, 167 
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negative, 96, 365, 308 
notation for, 86 
number of linearly independent, 292 
in principal-axes solution, 149, 473-91, 
497 
as EHRE on co-ordinate axes, 89, 
28 


reference abilities underlying, 68-69, 325 
as regression coefficients, 179, 504 
relation to correlation matrix, 95, 100, 

281 
single-common, 
270-81, 284, 289-90 
in Spearman's formula, 263 
specific, S6, 146 
unique, 78, 86 
zero, 89, 147, 181, 292, 324, 
355-56 

Factor matrix 
augmented oblique, 422-27 
centroid, 167, 170, 194, 231, 285-87 
common factors in, 73-86 
complete, 76-82, 86, 146 
consistent, 289-90. 
effect on 

of communalities 
of complexity, 3 
of factoring method, 363 
of selection, 4 168-60 
of treatment о! , 868-69 
74-70 147, 181-01, 
, 377-78 


273-76, 


340441, 


12: 


282, 312 


orn 
extended, 2 
factor patterns in; see Factor patterns 

nental equations for, 68, 71-73, 


geometr! interpretation of, 87-100 


imaginaries in, 286 А 
interpretation of, 95-96, 99, 177, 254, 
33 49, 3 


invi 

normalized, 131-33, 146 

notation for, 86 

number of independent par 
177, 291-94, 307, 311, 33: 

order of appearance of primary factors 


rameters in, 


for pal-axes solution, 477-92, 406- 

502, 509 

rank of, 92, 97-98, 343 

reduced, 76, 79-82, 86 . 

reduction of, to unit rank, 268 

reference frame of; see Reference frame 

relation of M 
to correlation matrix, 79-82, 92, 95, 

100, 281 

to factorial structure, 92 
to population matrix, 71 

rotated, 135-30, 202-6, 210-13, 228-29, 
253, 324, 326, 332, 334, 347-53, 369, 
378 

row of, 87 

second-order, 419 

sign reversals in, 96-97, 100 


Factor matrix—conlinued 
and simple structure, 311—135 
single-common, 2 8, 286, 288- 
90, 488 

specific factors in, 73-86 

test coefficients in, 71-73 

transformation from orthogonal to 
oblique, 347-54 

transpose of, 78-81 

unique factors in, 73-76, 79 

uniqueness of, 93, 280. 00, 343 


l'actor patterns 
characteristics of, 191-93 
corresponding to configurational types, 
88, 97, 182-90, 419, 441, 443 
defined, 191, 320 
effect of score treatment on, 369 
first- and second-order, 419 
role of, xi, 365 
in simple structure, 181-82, 190, 369 
Factor problem 
determinacy of, 81, 361, 378, 473 
formulation of, xi, 76 
multiple-, 283, 473 
non-linear, 
single-, 25 
starting-point of, 7 
steps in, 101 
types of, 340 
Factorial Analysis of Human Ability, The, 
by Godfrey Thomson, ix, 283, 200, 
148, 464, 465 
Factorial invariance; see Invariance 
Factorial Studies of Intelligence, by L. L. 
Thurstone and Thelma Gwinn Thur- 
stone, 402 
Factoring methods 


5 correlation hier- 
archy, 279 
centroid, 111, 149- 
centroid versus principal-axes solution, 
178 

criterion of validity of, 281, 375 
diagonal, 101- 
direct summation, 277-78 
graphical aids to, 407—500 
group centroid, 157-60 
grouping, 111-16. _ 
iterative, of Hotelling, 279, 483-85, 497, 

503, 505 : 
logarithmie solution, 277 
multiple group, 1703 

simplified centroid, 15% 
Spearman's single-factor formula, 273- 

6 
E E approximations, 278- 
tetrad differences procedure, 26 
for unit rank, 273-79 
Factors 


erage, 180 : А | 
Bipolar; 121, 182, 189-90, 193, 215, 363 
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causative, 435 

central, 180 

centroid, 152-53, 232 

common; see Common factors 

contribution to variance of, 68 SG, 340-41 

dependent composite, 190-01 

doublet, 182, 299, 306-7, 315, 363, 417- 
18, 441- 


first-order, 4 
general; see € 

order factors 
group; see Group factors 


high 14 
imagi 6 


incidental, 436-42, 469, 171 
linear combination of, 68 
nature of, 55-59 
orthogonal, 70, 73 
orthogong 


73, 85, 99 
ince of, 73 
e Second-order factors 


static, 345 
third-order, 414 
triplet, 182 
types of, 182 
uncorrelated, 139-10, 411 
unique; see Unique factors 
unitary, 343-14 
Factors of Mind, The, by Cyril Burt, ix 
Faculty psychology, x, 56, 70, 145 -16, 177 
Ferrar, W. L., 1 
First Course in Theory of Equations, by 
L. E. Dickson, 44 
Flywheel problem, 344-15 
Frisch, Ragnar, 514 
Functional unities, 57-58, 145-40, 284 
506-8; see also Primary factors 
Functions 
as analytical criteria for reference frame, 
377-78 
cognitive, 333 
linear, 54—55 


monotonic, 322, 368-69 
of parameters of factorial domain, 322- 
23 


score, 319-27, 365, 368 
Fundamental concepts, 328-29 
Fundamental criterion of valid factorial 

method, 361 
Fundamental equations, 68-86, 149 


Fundamental factor theorem, vi, 79-81, 
92, 149, 170, 259, 351, 
504-5 

Fundamental postulate, 63 


INDEX 


g-factor, vi, viii, 188, 271-73, 284, 421, 
430, 507 
Gaussian distribution, 61-66 
General factor 
configurations involving, 188-89 
defined, 121, 182 
differentiated from general common fac- 
tor, 130-31 
orthogonal, 
second-order; 
Spearman's y, vi, 
421, 439, 507 
1 intellective factor g; see General 
factor 


71, 273, 416-10 
second-order factors 


viii, ISS, 271-73, 284, 


Ger 


Generating parameters, 434-37 
Genetic I 
Geometrie: 
augmented co-ordinate: 
communality, 100, 
correction for uniqueness, 
icients, 89-91 


sis of mental traits, 333 


interpretation of 
340) 


factor loading, 89, 99 
factor matrix, 87-100 
linear equations, 33-38 
matrices, 36-37 


number of te to determine single 
common factor, 264-65 
primary factors, 319 
r<n common factors, 07-08 
reference factor, 99 
reflection of traits, 153-57 
residuals, 153-57 
score functions, 321-24 . | 
sign reversals in the factor matrix, 96- 
98, 100 „н 
simple structure, 96, 00, 130, 257-58, 
354-55 
single-common-factor loading, 270 
test battery, SU 
ests, 87, 00 
sformation 
oblique, 43 
orthogonal, 38-13 
zero factor loadings, 89, 135-36 


Geometrical models a 
three-dimensional, 125-48 
two-dimensional, 87-100 

Gottschaldt Figures test, 366 

Gramian matrix, 10, 304 

Gramian properties of correlation matrix, 

82, 282-83, 286, 280, 304 

Graphical methods 

aids to factoring, 407—500 — 

determination of hyperplane, 377 

determination of two planes 238 

estimating communalities, 306 

inspection of constellations, 344 _ 

isolation of simple structure, 349-50 
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iterative determination 
s, 407—500 
reference frame, 225-58, 377 
plotting of 
normalized vectors on sphere, 131-39, 
104 
three-dimensional sections, 195, 225- 
39 
two-dimensional sections, 194-224 
rotation of axes, 131-39, 205 


of principal 


Graphical representation of 
equation, 14-15 
factorial structure, 99 
proportional columns, 5 
simple structure, 257 
test configuration, 
tetrads, 270-72 

Graustein, W. C., 1, 41 

Great circles, 132, 134, 349 

Group centroid method of factoring, 157- 

61 

Group factors 
configuratio 
defined, 182, 4 
as disturbers, 273, 473 
and primary factors, 420-21 
projection of tests on, 422 

Grouping method of factoring, 111-16, 149 


s involving, 188-80 
1 


Heywood, H. B., 280 
Heywood case: 


Я 


Hotelling, Harold, xii, 27 
97, 503, 505-6 
IH y perplanes 
angular separation of, 340 
bounding, 343 
co-ordinate, 327-28, 
382-84 
defined, 328, 31243 
dimensions 
equation of, 
interpretation of 
normals to, 34— 
oblique, 349, 


2-91, 493- 


354-55 


347-49, 


216, 336, 338 


)0 - 

Positive co-ordinate, 343, 384 

secondary, 331 

in simple structure, 327-28, 331, 335, 
342-43, 5 

for the trait configuration, 327-28, 354- 


55 
Hypotheses in fac sis, 122-23, 
187-88, 216, 2 ~26, 344, 


506-8 
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Identity matrix, 22-23, 75, 416 
Independence 
defined, 63 
experimental, 63, 201, 313 
linear, 63; see also Linear dependence 
statistical, 6: 
Individual differences 
abilitie causes of, 52, 69-70, 325 
and identical test scores, 70 
parameters involved in, 58-50, 324 25 
at point age, 
study of, 52, 5: 
Intelligence, theo 
Inte 


"olumnar correlation. criterion, 260 
62 
Intercolumnar 
260-62 
Intercorrelations; see Correlation c ef- 
ficients; Correlations 
Interpretation 
of general intellective factor у, 273 
objective, 145, 338 
plausibility of, 336-40 
of primary factors; s ry factors 
of reduction to unit rank, 268 70 
of second-order factors, 414 
subjective, 145, 338, 506-8 
of trait configuration, 327 
Interval, score, 367 


proportionality criterion, 


Introduction to Algebraic Theories, by A. А. 
Albert, 1 
Introduction lo Higher Algebra, by Maxime 
Bocher, 1 
Introspection in factorial investigation, 
Invariance 
configurational, viii-ix, 364 66 
criterion of, 360 
of factor loadings, 360-76 
of factor pattern, 191 
factorial -76 
metric, 363-64 
1, viii, 363-64 
of scalar product, 91 
of subbattery of tests, 36: 
types of, 363-67 
of weights of primary abilities, 361 


Inverse 
computation of, 25-26, 46-48 
of a matrix, 23-26 
of produet of matrices, 2 
in regression equations, 
of transformation matrix, 137 

Isolation of 
common factors, 73-76, 8S1 
primary abilities, 73, 361, 506-8 
simple structure; see Simple structure 
specific factors, 73-76 
unique factors, 73-76 

Iterative method of factoring, 270, 482- 

500, 503, 505-6 
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Kelley, Truman L., viii, 287, 293, 300 
Kelley-Wood tables, 66, 368 
Kronecker's delta, 39 


Lagrange's method of undetermined multi- 
pliers, 475 

Law, scientifie, 51-55 

oroblem, 140, 504, 508 

Ledermann, Walter, 287 89, 444, 448 

Line 
equations of, 
slope of, 35 


t-squi 


33-34 
218 20, 400 500 
Linear combination of 
columns of lation matri 
factors, 1% ‚ 147 48, 352-53, 4 
504 
test scores, 511-12 
tests, 60, 44043 
Linear dependence of 
arrays of matrix, 32-33, 300-301, 314-18 
centroid vectors, 171 
columns of factor matrix, 82, 204, 291- 


94 
factors, 63, 69, 82, 282, 412-22 
planes, 238-39 
prim vectors, 420, 


reference vectors 


tes 
vectors, 48-50, 230 

33-38 
Linear functions, хі, 32-33, 54 


Loadings; 


Linear equations, 


55, 63, 69 


^e Factor loadings 

Logarithmic solution of single-factor prob- 
lem, 277 

Long reference vectors, 196-202, 234-38 


Matrix 
adjoint of, 9, 25 
array of, 2, 14 
cell of, 2 
characteristic, 27-28, 476 
column of, 2 
composite, 301-4 
correlation; see Correlation matrix 
covariance, 453, 455 
defined, 1 
determinant of, 3-13, 20 
diagonal, 21-23, 132-38, 35 
difference of two, 10 
element of, 2 
equation, 15-19 
factor; see Factor matrix 
of factor scores, 512 
formulation of factor problem, ix, 76, 

250-62 

geometrical interpretation of, 36-37 
Gramian, 10 
in Holzinger's method, viii 
identity, 22 
inverse of, 23-26, 46-48 


INDEX 


multiplication, 14-21 
associative, 19 
cross products in, 11 
distributive, 20 
non-commutative, 1% 
post-, 19-20 
pre-, 19-20 
summational notation in, 29-31 
non-singular, 11 
order of, 1-2 
orthogonal by columns, 41 
orthogonal by rows, 41 
population, 68, 71, 86 
positive-definite, 10 
rank of, 10, $ 
reciprocal of, 2: 
of regression weights, 512 


sealar, 
seore, 68, 71, 77, S6, 03, 324, 504-5 
sectioning of, 171, 301 4 

singular, 11 

skew symmetric, 10 

sum of two, 10) 

symmetric, 10, 46, 473 

of a transformation, 88-43 
transpose of, 3 

transpose of produet of, 16 

hine, 206 


Matrix-multiplying 1 
Maturation factor, 507 
Maximum variance criterion, 177 TS 


Mean of test scores 24, 368 


Measurements; sce Seores 
Medland, Frank, 318 
Memory factor, 145 
Mental age score, 442 


Method of extended vectors, 225-58, 106, 


458 
Method of minimizing weighted sums, 
396108 
Method of undetermined multipliers, 475 
Minors 


in estimation of communalities, 206-97, 


. 304, 308-0 
in evaluation o 
27-28, 476 
first, 7 
and ideal rank of R, 317-18 
principal, 6, 262, 286, 304 "y 
second-order, vi, viii, 265-68, 270-71 
as tetrads, 205-08 
for unit rank, 265-68, 270 
Models 
geometrical, 87-100 
spherical, 127-43 
vector, 125-27 
Modern Algebraic Theories, 
son, 1 » 
Multiple correlation, 59-60, 68, 17 
804, 511-16 


f characteristic equation, 


by L. E. Dick- 


9, 327, 
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Multiple group method of factoring, 170- 
nu 


Music factor, 257 


distribution of ability, 62-63, 325, 


Normal to hyperplane 
direetion ines of, 34-35, 226-27, 238- 
30, 319 
relation to primary trait vectors, 347- 
52 


and zero factor loadings, 323-24 


7 vectors, o 
in scientific arch, 
summary of factorial, 85-86 

Number of 

factors determined from x tests, 291-94 

factors versus number of tests, 82, 07-08, 

291-04, 307-11, 440 

tests to determine r factors, 201-04 

tes) 


for n tests, 265-70 
Number factor, 73, 145, 333, 514-15 


Observation equation, 68, 71, 325, 357-59 

Octants, 132, 141748 

Opposites test, 360 

Order , 
concepts of underlying, 55-56, 58, 258, 

436 

defined, 328 

of determinant, 3 

hierarchical, 273 

of matrix, 

oblique simple, 321 

orthogonal simple, 321, 328 

and rank, 82 0, 484 
simple, 320-21 i 

Orthogonality 
of matrix, 
restriction of, vii, 189 40 
of test vectors, 91 
and zero factor loadings, 

Osgood, W. F., 1, 41 

Otis-Kelley formula, 449-50 


89-91 


Parallel tests, 83 

Parameters т 
of abilitie 08 ap, p19 27 
alternative, 331-2 

Я ive factors, 484-35 

xi, 333-34 

complexity as number of, 320 


correlated, 139—10, 320-21 
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Parameters—continued 
in correlation matrix, 291-94, 307 
defined, 331, 414 
and degrees of freedom, 52 
of domain, 180, 291-9 
471-72 
in factor matrix, 177, 291-94, 307, 311 
generating, 434-37 
geometrical representation of, 127 
25 
identification of, 176-77, 
incidental, 436-39 
linear dependence in, 412-16 
in linear equations, 34-35 
primary factors Primary factors 
second-order, 411-16 
of set of scores, 319-25 
uncorrelated IO, 320-21 
unique, 320, 2 33, 412 
Partial correlation formula, 448-49 
Percentile ranks, 64-66, 368 
Personality schedules, 322 
Personality traits, 322-24 
Plane and Solid Analytic Geometry, by 
W. F. Osgood and W. С. Graustein, 1 
41 
Planes 
adjustment of, in rotation, 241-51 
best-fitting, for score surfac 3-24 
ch on independence of, 238-39 
co-ordinate, 181, 225-28 
equation of, 34 
trace of; see Trace 
Plotting of 
normalized vectors on a sphere, 131-39, 
104 
oblique reference axes, 206 
three-dimensional sections, 19. 
two-dimensional sections, 194-4 
Polynomial, roots of, 44—15 
Population 
equivalent, 360-61 
experimental, 62, 324-26, 367, 458 
general, 325, 367 
selection of, 55, 324-25, 360-61, 438, 
440, 457-58, 470—72 
statistical, 62 
Population matrix 
for common factors 
for correlated primary | 
elements of, 68, 71, 356 
notation for, 86 
oblique, 356-59 
relation of de Et 
to factor matrix, 71, 77-78 
to score matrix, 71 
transpose of, 77-78 
Positive manifold, 215, 217, 341-43, 397 
Positive octant, 132, 147-48 
Positive region, 342 


324, 365 


, 


actors, 356-59 


2, 320, 331-33, 


Primar 


319- 


Prediction 

factor a and empirical, 59-62, 68, 
16 

rial composition of new tests, 

33, 366-67, 506 

and postulated factors, 338, 366-67 

abilities (see also Abilities; Pri- 


ince of weights of, 361 

tion of, 73, 361, 506-8 

zdle-factor matrix, 268 

factors (sce also Common factors) 

ıl of individuals as to, 62-63, 

: 7, 511-16 

and constellations, 344-4 

correlation between, 78, 1 
4 


5 
3T 
1 


228-20, 320, 351-59, 
170 


lation of test with, 327 

d а for acceptability of, 336-40. 
defined, 348 

effect of selection on, 5 
exploratory studies of, 
7, 319-21, 33: 


5 
genetic basis of, 
and group facte 
incidental, 436-39 
interpretation of, 


138-41, 144-46, 2 

40, 378, 4: 
invariance of, 
isolation of, 
8, 3. 
linearly independent, and rank, 82, 282 
naming of, 56, 145 
number N, 145, 333, 514-15 
oblique, 63, 
order of appearance in faetor matrix of, 

255 

orthogonal, 63, 
as parameters, 

2 


j, 208, 33: 

i10, 470-71 
360-76 

7, 134, 179-81, 325 


5-59, 332-33, 471- 


positive, 215-16 

predictability of postulated 

predietion of scores on, 511-15 

relation of, to factor matrix, 71-73, 334— 
36, 347-54 

and second-order fac 

space S, 145, 324-26, 51 

theory of, 55-67, 319-29, 345-46 

uncorrelated, 349, 411, 416 

uniqueness of, 329, 332 -35, 469 


5, 411-39 


unit 34445 
validity of 4 
verbal V, 508, 


514-15 
word fluency W, 73, 338, 508 


INDEX 


348-54 
ce also Primary 


Primary trait vectors 
Primary traits, 347 
factors 


Primary vectors 
related, 411, 421 

cosines of, 111 
defined, 137, 348 
direction cosines of, 137 
notation for, 348 
relation of 

to correlations, 3. 

to group factor: 

to hyperplane 

to test vectors, 
in second-order domain, 4 
of unit rank, 421 


Principal axes 
defined, 478 
direction cosines of, 474 

‘ation of, 177-78, 362, 

500 

major, 6, 362-63, 415, 474, 478, A86- 
$7, 503, 505-6, 508 

mean, 478, 487, 503, 500 

orthogonal, 477, 508 


7 
173-77, 480 


as reference frame, 177-78, 284-87, 362, 
173—510 

relation of, to centroid axes, 149, 177- 
79, D 


ion to, from orthogonal ref- 
474-77, 506 
Principal-axes solution 
advantages of, 503-10 
applicability of, 284-87 
10 
and centroid method, 177-79 


tra nator 
erence frame, 


206, 474, 503- 


characteristic e ut in, dM 45, 476- 
7 А, 487-91, 5 3 

characteristics of, 177- 74 

defined, 474 

diagona il entries in, 474, 484, 509 

direct, viii, 480-1 

factor loadings i in, ‚140, 

ЖО 1 

of Hotelling, 279, 482 , 503, 505-6 


indirect, 473-80, Күй 
tive method in, 207, 482- 
505-6 
of Kelley, viii 
squares solution, 140 
ions of, 216, 503-10 
maximizing of factor loadings in, 
maximum-variance criterion in, 1 
numerical example of 
direct, 484-85, 491 
indirect, 477-80 
properties of, 500-503 
rank in, 484 
residuals in, 493-97 
npling-error formulae for, x 
simple-structure solution versus, 503 
squaring process in, 485-02 2 


500, 503, 


or 
Па 
Бә] 


thecrems pertai ding to, 500-503 
theory of, 47 
uniqueness of, 178, 474, 503 

Principal components 
of Hotelling, 482, 484, 486 
‘Tucker's method of determining, 497 

Probability curve, 66 

Probability tables, 368 

Profile of ability, 511 

Projections of test vectors 
on centroid axes, 151,217-19, 

207-98 

heck of inde pendence of planes, 238- 

9 
defined, 94 
into an ellipsoid, ч 
on group factor, 
maximizing sums of squares of, 173-7 -74 
minimizing weighted sums of, 377, 396- 

408 
normaliz 
nume 
on principal ax 
properties of, 5 
on reference axes, Sf 
on second order 1 uet 4 
on single-factor 
zero, 197, 3: 

Proportionality 


15, 448 


of, 363 


-63, 473-01 


rion | of Шок AA 260-62 
of two variables, 291 


338 


Psychology, science of, 


Quartile range, 368 


Random configuration, 182-83 
Rank 
of correlation ma atrix 
complete, 
reduced, ¢ 
14, 456, - 
defined, 10, 282 
effect on 
of diagonal values, 282-87, 296-307, 
484 
of modifying raw scores, 323 
of selection, 108, 470-7 
of factor matrix, d 
of Gramian matr 
in Hey wood case, 289- 90 
ideal, 317-18 
of matrix of primary factors, 412, 434, 
436 
of matrix of raw scores, 4 
and minimum trace 
and multiple solutions for communal- 
ities, 309 
as number of independent factors, 291 
94 


269, 343 
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Rank—conlinued 
and order, 82 
reduction in, 2 


)-40, 269 


of second-order domain, 411-14, 417-22 


of the side correl: s, 283 


see Unit rank 
zero, 260 


unit; 


Rank one; see Unit rank 
Rank order, 64, 367-69 
reference abilities 
correlation between, 78 
defined, 60 
in factorial formulation, 68-76, 78 
notation for, 85 
oblique, 60 
orthogonal, 60 
postulation of, 51-59, 62-67 
relation of, to factor loadings, 73 
teference axes 
Reference vectors) 
defined, 195 
determined by score surfaces, 321 -22 
independence of, 90-91 
location of 
with respect to centroid, 
by unitar "tors, 344 
oblique, 135-39, 194 205, 
orthogonal, 38. 43, 89, 1t 
in principal solution, - 
rotation of; see Rotation of axes 
teference factors, 71-73, 85, 99 
Reference frame ( see also Reference axes: 
Reference vectors) 
arbitrary, 96, 119-24, 
choice of, 1 15, 284-85 
and correlated factors, 139 40 
criteria for location of. 
analytical, 356, 377-78 
graphical, 108, 334-35 
statistical, 177-81 
determined by centroid 
361-62 
effect on 
of factoring method, 363 
of sign reversals in /-matrix, 96 
indeterminacy of location of, 473-74 
insertion of, into configuration, 92, 98, 
150-53 ^ — 
location of, vii, 
377-78 
oblique, vi-ix, 43, 
421 
orthogonal, 
in principal- 
relation to 
of correlation matrix, 92 
of factor matrix, 78, 91-92 
rotation of; see Rotation of axes 
in simple structure, 327-2 


Reference traits, 54-55, 139 


134, 327 


method, 327 


176-81, 321, 331-35, 


135-39, 215, 323-24, 


130, 251, 327 
473-510 


13, 137, 
s solution, 


MULTIPLE-FACTOR 


(see also Reference frame; 
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teference vectors (see also Reference axes; 
teference frame) 
adjustme nt of, 196 
bipolar, ¢ 
centroid 
choice o 
cosines 
direction 
221-2 
long, 1t 
positive. 
primary 9, 422 
relation ‘of, to hyperplanes and primary 
vectors ‚ 193, 31749 
residual, 
trial, 378-85, 306 401 
unit, S7, 99, 135, 195-6, ЗХ 
Reflection of test vectors, ¢ 
57, 165-66 
Regression equations 
in iterative procedure, 499-500 
for prediction problems, 59-62, 68, 179, 
504, 511-16 
for r on n, 499-500 
for s on z, 515-16 
for slope of regression line, 400 
weights in, 50-61, 511-16 
for x on s, 511-15 


132-38, 
5l 


193-96, 


teliability 
coefficient, S3-86 
as diagonal entry, 83, 481 
ıd error variance, 84 
notation for, 85-86 
perfect, 474 
relation to communality, 83-85, 282, 
287 
of test, 
Hesiduals 
centroid-factor, 153-57, 163 65, 178 
in diagonal method of fae 
effect of reflection on, 1537 
gencral-faetor, 473 
geometrical interpretation of, 155-57 
minimizing of, 60-61, 82, 149, 177-78, 


473-74, 


83-86 


pal-factor, 481- 


493-97 


177-79, 


‚ 149, 153-57, 105- 


Robinson, G., 45 


Roots 
of characteristic equation, 44 


» 477- 
79, 483-81, 487-91, 500-503 
of polynomial, 4445 
totation of axes 
adjustment of planes in, 241-51 
check on linear independence in, 239-40 
computational formulae for three rota- 
tions, 205 
computational procedure for, 
corrections in, 208-12, 248 


494-97, 


194-216 


INDEX 53 


and entries of factor matrix, ix, xi, 181- 
82, 292 

invariance of scalar product under, 91, 
99 

in method of extended vectors, 
377, 406 

methods of 
double-group, 408-10 
minimizing weighted sums 
by plotting on sphere, 131-3! 
single-plane, 216-24 

successive approximation, 378-06 
e three-dimensional sections, 


241-58, 


193, 


194-95, 


by “two-dimensions ul sections, 


oblique, - 39, 194-224 
orthogonal, 38-43, 108-10, 
from orthogonal reference fi 
cipal axes, viii, 286, 47 T 
purpose of, 101, 197, 292, 325-26 
to simple structure, 181-82, 
3, 508 
¢ of cosines and choice of, 241 
in three dimensions, 131-30, 104—224 
in two dimensions, 108-10 ۹ 
use of principal-factor matrix in, 509 


Row vector, 14 


S-matrix 
in rotation of ie Is 
of scores, 68, 71, 77, 86, 93, 
Sampling errors, X, "960- 61, 264, 267-68, 
283, 361 
Sampling theory, xii, 269 
Saturations; see Factor loadings; Projec- 
tions of test vectors 
Sealar, 23 
Scalar product of test vectors 
correlations as, 89-91, 449 
defined, 37 
notation for, 380 
theorems on, 91, 94 
Scale 
of direction cosines, 
normalized, 367-69 
Science 
fundamental principle o 
ideal constructs in, 51 
nature of, 51— 
of psye hology, i 
unique solution in, 332 
Score functions, 310-27, 365, 368-72 
Score matrix 
element of, 68 
factorial analysis of, 68-71, 504-5 
notation for, 71, 86 
relation of 
to correlation matrix, 77, 93 
to factor matrix, 71 
to population matrix, 71 
anspose of, 77 


137 


291 


Score surfaces, 321-24 

Scores 
composite, 
condensat 
defined, 6: 
derived, 63 
deviation of, 
functions for, 
T 

s linear combinations of seores on cor- 
related primary f: 

lized, 64-66, 367 


327 
m of, 177 79, 
73 


503-8 


323 
68-69, 319-27, 365, 368- 


тарле, of 'ovigi inal, 284 
selected for factorial study, 319 
standard; s cores 
standard deviation of, 368, 
summation, 442—143 
transformation of, 
true factor, 511 
true test, 62 

iance of, 86 


Scoring formula, 367 


443-50 


367-76 


Second-order factors 
defined, 411 
"immatie 


representations of, 416- 


domain of, 411-14, 420-22 

factor pattc for, 418-10 

general, 421-34 

interpretation of, 415, 434, 436 

notation for, 411-14 

number of linearly independent, 414 

occurrence of, vii-viii, 411 

neters of, 411-16, 434-39 

al example of 
four-dimensional, 427-34 
three-dimensional, 415-16 

relation to first-order factors, 411-39 

and tetrac 

theories of, 

transformation for, 


421-27 
Sectioning of matrix, 801—1 


Sections 
three-dimensional, 195, 2 
two-dimensional, 194 


Selection 
complete, 470 
E of test scores and, 441-43 
effect of 

on communalities, 
on correlations, 44 
on dispersions, 443-47, 45 
on factorial results 2 
360-61, 412-13, 438, 440 

on primary factors of, 456-6 
on simple structure of, 456 
geometrical representation of, 


144-45 
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Selection—continued 
multivariate, 443-14, 464-72 
partial, 471 
total, 471 
Otis-Kelley formula on restricted, 449- 
50 
partial, 470 
restr 
tests, 443-5: 
total, 470-71 
types of, 440 
univariate 
complete, 443 
partial, 443 )-64 
variable, 44 58, 464, 470 
Self-correlation of vest, 06-07, 
Sign-changing methods, 06-07, 140, 153- 
57, 165-66 
Simple configuration, 58, 328; see also Sim- 
ple structure 
Simple order, 320-21, 328-29 
Simple structure 
alternative, 329-31 
best-fitting, 355 
bipolar, 189-93 
complete, 361, 363, 378 
concept of, v, 58, 181-82, 319-46 
configurational types of, 181-03, 507 
criteria for, 334-40, 355-56 
defined, 181, 328 
determinacy of, 361, 363 
effect on 
of selection, 440 
of treatment of se 
equation of oblique, 354-55 
geometrical interpretation of, 327-2! 
graphical indications of, 247, 256 5 
335, 349 
hyperplanes of, 3: 
incomplete, 216, 
interpretation of 
and invariance, 7 67, 456 -61, 469 71 
isolation of, 181-82, 327—34, 343 
]ocating reference frame for 
analytical, 356, 377-78 
graphical, 247, 257-58, 3: 
340 
oblique, 328, 35 ) 
orthogonal, 247, 3-20, 352 
overdetermination of, 182, 216, 320, 361, 
363-64 
positive, 342 e Я 
and positive manifold, 341-44 E 
principal-axes solution versus, 503, 508- 
10 
purpose of, 332-33 
in r dimensions, 327-2 
rotation to, 181 i 3 
subspaces in, 827-28, 347, 377 
uniqueness of, 329-40 
vector model of, 126 
Simplified centroid method of factoring, 
159 


456, 459-64, 470 
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Simplified Multiple Factor Method, by L. L. 
Thurstone, 159 
Simultaneous equations, 13-18, 291, 476 
Single-common factor, 250-81, 284, 487- 
91 
Single-factor formula of Spearman, 273- 
76, 279-81, 284, 305 
Single-factor method 
ambiguity in, 268 
of direct summation method, 277-78 
limitations of, for multiple-factor anal- 
ysis, 279-81 
rithmic solution for, 277 
actor formula in, 273-70, 284, 


278-79 


ve approximatior 
tetrad differences procedure 
Single-plane method of rotation, 2 
Sisam, C. H., 1 
Skewness, 64, 367 
Slope of a line, 33, 499-500 
Snyder, V., 1 
Solid Analytical Geometry and Determin- 
ants, by Arnold Dresden, 1 
Space 
augmented, 421-22 
-factor; see 


Common-factor 


commor 


ictor, 73-76, 81-86, 146 
total factor, 176 
total test, 98, 176, 324, 420 
unique-factor, 73-76, 
Space factor, 147 
Spearman, Charles, vi 


515 


188, 259-76, 289- 
), 473 
ance, viii, 75, 81-85; see 
also Specificity 
ic factors 
absence of, 81 
change from common to, 361 
contribution of, 79 
defined, 73 
5 ation of, 287 
of Spearman, 272-73 
in total variance, 73-76 
Specificity 
defined, 75 
estimation of, 84-85 
in factor matrix of unit rank, 268 
notation and terminology for, 85 
separation from error variance of, 84, 
287 
Spherical model, 125-43 
Spherical triangle, 130, 134, 
364 
Square root, calculation of, 45-46 


147-48, 344, 


Squaring process in principal-axes solu- 
tion, 484-97 


INDEX 


Standard error, 261, 207-68 
Standard scores 
in abilities, 63, 70, 511-12 
in correlated primary factors, 37 
distribution of, ( ‚ 368 
linear form of, 68-69 
matrix of, 68, 71 
normalized, 64, 368, 448 
notation for, 8. 
reduction to, 64-66, 368 
surfaces for, 323-2- 
in tests, estimated from standard score 
in ability. -16 
in unitary factors, 343 
Statistic, 414 
Statistical Confluence Analysis by Means of 
Complete Regression Systems, by Rag- 
nar Frisch, 514 
Slalistical Résumé of the Spearman Two- 
Factor Theory, by Warl J. Holzinger, 
267 
Stephenson, William, 271 
Street Gestalt Completion test, 333 
Stretching factor, 64, 132, 324, 381, 448, 
1 


oblique, 347-59 
simple; see Simple structure 


Subjects of experimental population, 324- 
26, 360-61, 440, 470-72 
Subspaces, 327-28, 347, 340, 377 
Nuecessive approximation method of rota- 
tion, 378 -96 
Successive approximation method for unit 
rank, 278-79 
Summational procedures in c 
centroid method; see Centroid method of 
faetoring 
determination of single-common factor, 
277-78 
estimation of communalities, 304-5 
grouping method of factoring, 111-16, 
149 
Surfaces, score, 321-24 
Syllogism test, 326 
Synonyms tests, 338, 508 


Temperamental traits, 322-24 


lest batteries 
as exploratory domains, 
Я 06-8 EUG 
invarianee in, 332-33, 361-67, 


57-58, 68-70, 


factoria 

. 373-76 

indeterminacy in, 3 
l'est coefficients, 68-69, 71-73, 86 


"Test score; see Scores 
Test vector: 
angular separations of, 03, 344-15 
centroid of, 151—5 
configuration of; see Configuration 
constellations of, 184-85, 297-99, 301, 
314, 344-45 
defined, 87 
equation for, 148 
extended, 225-58 
geometrical models 
three-dimensional, 125-48 
two-dimensional, 87-100 
length of, 91, 99 
as linear combinations of 
primary vectors, 127-30, 35: 
reference vectors, 14748 
normalized, 131, 379, 401, 448 
null, 175, 181, 3: 59 
plotting of normalized, 131-89, 194-95 
in positive region, 342 
projections of; see Projections of test 
vectors 
properties of, 87-07 
reflection of, 153-57 
sealar product of, 89-91, 90 
in second-order domain, 422 
in simple structure, 1S1, 338-39 
unit, 91, 94, 127-28, 131 
‘Tests 
complexity of, 320, 326, 328, 333-35, 
351 
composite, 69, 268, 327, 514-15 
correlation between, 57, 63, 76-82, 89- 
91, 95 
correlation with primary factors of, 327 
defined, 62 
effect of selection of, 364, 440 
factorial composition of 
invariance of, 325-26, 333, 360-63, 
372-75 
prediction of, 333, 366, 506-8 
geometrical interpretation of, 87, 99 
notation for, 86 
number, 73, 145, 333, 361-62 
number of, to determine common fac- 
tors, 82, 97-98, 262-65, 291-94, 307- 
11, 440 
parallel, 83 
parameters of, 325, 332-33 
personality, 322-24 * 
primary traits in terms of, 348 
psychological nature of, 826 
pure, 351-52 
ТЕР y of, 83-86, 286, 474 aor 
with saturations in several factors, 326 
327 
scores on; see Scores 
selection, 443 aegis 
space, 145, 324-26, 361-62 
in tetrad difference equation, 266 
use of, 325-27 
validity coefficients of, 327, 516 
variance of; see Variance 
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Tests—continued 
verbal, 145, 333, 338, 361-62 
weights of primary mental abilities for, 
69, 361 


Tetrad differences procedure 
characteristics of, 265-68, 274, 293, 430 
diagonal elements in, 266, 284 
equation for, 26. 56, 273, 293 
matrix interpretation of, 266, 270 

Tetrads 
evaluation of, 
graphical analysis of, 270-72 
number of, 265-70 
relation of, to second-order minors, vi, 

266-67 
of subgroup i in correlation matrix, 271 
vanishing, 266-68, 270-71 


Theory of Multiple Factors, by L. L. 
stone, 175, 273, 277, 282, 473 
Theory of single common fac мог, 272 
Theory of two factors, 271-73 
Thomson, Godfrey H., ix, 269-70, 
287-01, 443-66, 470 
‘Time limits, 367 
Trace of 
correlation matrix, 287-80 
plane, 197, 206-10, 226-29, 232 ff. 
"Trait configuration, 327-28, 3: 
45 
Trait vectors; see Test vectors 
Traits (see also Tests) 
clusters of, 126-27 
configuration of, 327-28, 334-35 
correlation between, 146, 322 
defined, 62 
description of, 54-55 
descriptive categories as, 328-29 
genetic basis of mental, 333 
кешпей! да interpretation of primary, 
349-50 
personality, 322-24 
primary, 347-52 
reference, 54-55, 139 
temperamental, 322 
Transformation 
determinant of, 40 
direction cosines for, 38-43 
for group factors, 424-26 
infinitesimal orthogonal, 343 
linear, 38, 301, 368 
matrix of, 38, 43, 134-39, 347 


monotonic, 369 

oblique, 43, 131-39, 194-224, 347-52, 
422-26 . 

orthogonal, 38-43, 109-10, 136-37, 202, 
231, 343, 352, 424-25 

from orthogonal reference frame to prin- 
cipal axes, viii, 286, 474—77 

for second-order factors, 422-27 

of test scores, 368-69 


268, 270 


Thur- 


283, 


MULTIPLE-FACTOR ANALYSIS 


Trapezoid population, 427-34 
Triangular configuration, 184-87, 192 
Triplet, 182 


Tryon, Robert, 344-15 
Tucker, Ledyard | R., xiii, 16, 351, 356, 358, 
441, 450, 


"Turnbull, IT. W., 1,512 
Two-factor theory: 267, 271-73 


Unique factors 
and common factors, 73-76, 272-73 
contribution of, to dingon: id cells, 70 
defined, 182 
notation for, 86 
postulation of, 73-76 
und unity in dingonals, 281, 286 


Unique variance 

absence of, 177 

determination of, 286-87 

effect of diagonal values on, 282-87 

error variance in, 74, 84 

experimental depende nee and, 12, 
468 

in Heywood case, 289-90 

separation of common-factor 
and, 282 

specific variance in, 74, 84-85 


variance 


Uniqueness 
and centroid method, 161 

of communalities, 307-11, 317-18 

of configuration, 92-93, 96, 178-80, 203 
correction for, 146- 47, 268 

of factor solution, 98, 178, 180 

of parameters 23 

of primary f 


T р 
хез A AA 178, 474, 503 
le structure, 329-31, 331-40 

ee also Unique variance 
complement of, 282 

defined, 75 

notation for, 85 


Unit of measurement, 367-68 
Unit rank 


correlation matrix of, viii-ix, 250-81 
diagonal entries and, 259, 262, 274, 277, 


"hes matrix for, 259-62, 268, 285-86, 
289-90, 488 
factorial reduction to, 268-70, 285-86 
factoring methods for, 206-68, 273-79 
Heywood case and, 289-91 
limiting case of, 261 
minimum trace and, 287-89 
number of tests for, 262-65 
in second-order domain, 412-14, 420-22 
submatrices of, 171 
theorems on, 260- 62, 269, 290 
Unitary factors, 343-44 
Univariate selection, 365, 443-64, 470-72 


Validity coefficients of tests, 327, 516 
Variables 
criterion for, 4 
dependent, 59 
516 
in factor analy: 
independent, 5t 
504, 511, 513, j 
selection of, 443-53, 458, 464, 470 
unitary factors in test, 343 
Variane 
of sco! 
of a test 
common-factor; see 
variance 


7-58, 313, 329 


86 


contribution of factors to, 73-76, 340— 


41 
effect of selection on, 44 
error-factor; see Error 
notation for, 85-86 
single-common-f: 
specifie-factor, v 


50 


г, 207 
i, 73-76, 84-85 


in terms of communality and unique- 


ness, 75, 85 
total, viii, 73-76, 85-86, 
unique; see Unique variance 
unit, 269-70, 466 
Veetor models, 125-28 
Vectors 
centroid, 175, 217-19, 225, 251, 327 
column, 14, 422-27 
correction, 879-82 
extended, 227 " 
linear dependence of, 48-50 
in a matrix, 14 
models of, 


68, 116, 179, 201, 


Common-factor 


эг variance 


176, 505, 508 


INDEX 


primary; see Primary vectors 
primary trait, 348-54 
reference; see Reference vectors 
row, 14 

scalar product of two, 37, 89 

, 87; see also Test vectors 
trait; see Test vectors 

trial, 378-85 
volume, 415 


Vectors of Mind, The, by L. L. Thurstone, 


v-vi, 283, 327, 344, 360-61 


Verbal factor, 145, 333, 338, 361-62, 508, 


514-15 
Visual perception, 55, 507 
Visual tests, 324, 506 
Visualizing factor, 73 


Weights 
of abilities in a test, 69, 361 
in factor analysis, 60-61 
in multiple group method, 173 
regression, 59-61, 511-14, 516 
of test vectors, 148, 35: 
Whittaker, E. T., 45 


Wishart, John, 267 

Wolfle, Dael L., ix 

Worcester, Jane, 263, 309-11 
Word flueney factor, 73, 338, 508 


Zero 


Wilson, E. B., 263, 293, 309-11, 318 


correlation, 91, 139-40, 321, 341, 435 


entries in fact 
292, 324, 335-36 


r matrix, 89, 147, 181, 


factor loadings, 89, 147, 181, 292, 324 
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